I have a monthly Spark job that process data and save into Hive/Impala tables (file storage format is parquet). The granularity of the table is daily data, but source data for this job also comes monthly job.
I'm trying to see how to best partition the table. I'm thinking of partitioning the table base a month key. Wondering if anyone sees any problems with this approach, or have other suggestions? Thanks.