Does REFRESH table command refresh the metadata in Impala when a partition location is changed in Hive?
I am changing the Hive table partition location using
ALTER TABLE db.table partition(key=value1, key2=value2) set location='path'
After that, I am running REFRESH db.table in Impala which is not updating metadata. If I run INVALIDATE METADATA it is working.
There is Impala JIRA-4364 open for that. However its been in product backlog since 2017, so currently INVALIDATE METADATA is the only workaround.
UPDATE: This has been fixed in Impala 4.0 (see same JIRA link above).
Please make sure you perform msck repair after loading into the Hive partition instance.
Afterwards, you can invalidate the metadata for the DB in which the table resides in Impala shell/UI