enableHiveSupport throws error in java spark code

Viewed 2815

I have a very simple application that is trying to read an orc file from /src/main/resources using spark. I keep getting this error:

Unable to instantiate SparkSession with Hive support because Hive classes are not found.

I have tried adding the dependency

<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.11</artifactId>
<version>2.0.0</version>
</dependency>

as recommended here: Unable to instantiate SparkSession with Hive support because Hive classes are not found

however, no matter what I have added, I still get this error.

I am running this on my local windows machine through NetBeans IDE.

my code:

import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.*;

public class Main {

    public static void main(String[] args) {
        SparkSession spark = SparkSession
                .builder()
                .enableHiveSupport()
                .appName("Java Spark SQL basic example")
                .getOrCreate();

        Dataset<Row> df = spark.read().orc("/src/main/resources/testdir");        
        spark.close();
    }
}
1 Answers
Related