I have a PySpark code which develops the query and runs insert into command on another Hive table which is internally mapped to a HBase table. When I run the insert into command onto the Hive table using spark sql I get the following exception..
java.lang.ClassCastException: org.apache.hadoop.hive.hbase.HiveHBaseTableOutputFormat cannot be cast to org.apache.hadoop.hive.ql.io.HiveOutputFormat
I checked the datatypes and tblproperties but unable to get through this exception.
The versions I am using are:
PySpark -- 1.6.0
Hive -- 1.1.0-cdh5.8.2
The table properties are:
ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe'
stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
with serdeproperties("hbase.columns.mapping"=":key,colf:a")
tblproperties("hbase.table.name"="abc",'hbase.mapred.output.outputtable' = 'abc');
I tried removing the Row Format Serde even though getting the same issue..
Am I getting the issue because of the versions not getting matched?? or am I going wrong??