Here are the files stored by EC
Found 1 items
-rwx--x--x 1 hiidoagent hdfs RS-6-3-1024k 3846 2022-09-08 21:28 hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220907/event_type=listStatus_par/part-00000-09410305-ccd9-4295-b36e-b7ab1e38d3a4.c000
Found 1 items
-rwx--x--x 1 hiidoagent hdfs RS-6-3-1024k 5737 2022-09-08 21:22 hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220907/event_type=listStatus_tab/part-00000-56a03378-bc29-42e1-ac59-8b145a9dc092.c000
Found 1 items
-rwx--x--x 1 hiidoagent hdfs RS-6-3-1024k 6078 2022-09-08 21:27 hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220907/event_type=open/part-00000-3e1a11b3-2d52-4d0e-99ab-aa0c391ce847.c000
hdfs dfs -du -h hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220907/*/
3.8 K 15.0 K hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220907/event_type=listStatus_par/part-00000-09410305-ccd9-4295-b36e-b7ab1e38d3a4.c000
5.6 K 22.4 K hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220907/event_type=listStatus_tab/part-00000-56a03378-bc29-42e1-ac59-8b145a9dc092.c000
5.9 K 23.7 K hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220907/event_type=open/part-00000-3e1a11b3-2d52-4d0e-99ab-aa0c391ce847.c000
How is this total size calculated, for example 5.9*?=23.7?
RS-3-2-1024k 0 2022-09-13 17:55 hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220912
hdfs dfs -du -h hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top
15.2 K 45.6 K hdfs://yycluster06/hive_warehouse/freshman.db/ec_hive_hot_data_top/dt=20220912
What IS THE MEANING OF EC FOR SMALL FILES, WHICH IS THE SAME SIZE AS 3xReplication, AND WHAT IS THE FORMULA FOR THIS?
3 xreplication 15.2 * 3 = 45.6
Rs-3-2-1024 K 15.2(Data blocks)+2(Parity Blocks)*15.2=45.6?