How to ingest orc in druid from base path of a hive external table?

user_8580485 0
Viewed 156

I have a hive external table pointed at location = "hdfs://localhost:8020/sample/path/" here /sample/path contains various partitions like

/sample/path/cola=123/colb=456
/sample/path/cola=324/colb=432
/sample/path/cola=322/colb=234

I have tried to ingest data into apache druid using index_parallel , while doing so i have to mention complete partion dirs upto leaf level :

"paths":"/sample/path/cola=123/colb=456,/sample/path/cola=324/colb=432,/sample/path/cola=322/colb=234"

Value for these partition columns is lost once they are ingested into druid

Ques : Is there some way i could specify the base path and retain value of partition columns after data ingestion ?

druid orc hdfs olap hive
1 Answers

I'm afraid not. You're ingesting the files, and they simply don't contain values for the partition column. To ingest this data you'll have to have the column in your table twice, once as partition column and again as a regular column.

user_1609640
Related