I create some dataframe with spark daily, and save it to HDFS location. Before saving I partition data by some fields, so path to data looks like this:
/warehouse/tablespace/external/hive/table_name/...
table_name directory has partitions like:
table_name/field=value1
table_name/field=value2
I create external table to operate the data with Hive and set location to data path. Each day I want to change location to new data path. But if I use ALTER TABLE table SET LOCATION 'new location', quering still return old data because partition's locations don't change.
Is there any way to tell Hive to search partitions in new location, without changing it one by one?