Recently I have created a delta table with a partition column created by generated column.
Below are my two observations. Nee your guidance.
1st Approach:
run_date DATE GENERATED ALWAYS AS (CAST(date_trunc('month', run_timestamp) AS DATE))
2nd Approach:
run_date DATE GENERATED ALWAYS AS (CAST(run_timestamp AS DATE))
When I created the partition column using the first approach, partition skipping is not working, but partition skipping is working for the second approach.
The first approach suffices our requirement. According to Databricks documentation value of the generated column can be decided based on a user-specified function over other columns in the Delta table.
But I am not getting in our case why partition skipping is not following. Am I missing any configuration in order to get the expected outcome?
Can you please guide me?