Hive Increase in bucket count- how the existing data split occurs to new buckets?

user_14153479 0
Viewed 33

In hive,I have Orc file formatted table with 10 buckets and the table has 1Tb of data already.If i increase the bucket count,will my existing data split occurs between new buckets automatically or do I need to reload the data in table . Is there any way to alter the bucket count? I am newbie to bucketing concepts.can someone help on answer this question?

hiveql hdfs apache-spark hive hadoop
1 Answers

If you use ALTER TABLE mytable CLUSTERED BY (my_field) INTO 10 BUCKETS, existing data will not be redistributed.
And new rows will be bucketed/redistributed into new buckets.

If you want a clean method, please follow -

  1. Create a new table with new structure.
  2. Insert data into the new table from old table.
  3. Drop old table.

This will redistribute whole data into new buckets.

user_64305230
Related