I have a Spark DataFrame with one of the columns as Vector type.
When I create a hive table on top of it, I don't know which type it is equivalent to
CREATE EXTERNAL TABLE mix (
topicdist ARRAY<DOUBLE>
)
STORED AS PARQUET
LOCATION 's3://path/to/file.parquet'
The table creation seems to work and return OK, but when I try
select topicdist from mix limit 1
The error I get:
Failed with exception java.io.IOException:java.lang.RuntimeException: Unknown hive type info array<double> when searching for field type