I have data like this in a dataframe in PySpark.
I need output like this
The logic goes like this - from table 1, the first date of category B for id=1 is 08/06/2022 and the first date for category A is 13/06/2022.So, for any dates on or after 13/06/2022 should have both categories A and B until the deleted date. So, for 08/06/2022, there is category B only and for 13/06/2022 there is category A and B. For 24/06/2022, there is just category A in table1 and the deleted date is 25/06/2022 but the output should have category B too as the first date of category B is 13/06/2022 and for 26/07/2022, there is just category B in table 1 and the output should have only category B for 26/07/2022 as category A was deleted on 25/06/2022. If there is no deleted date both categories should be there. If there is a deleted date both categories should be there until deleted date and the category with the deleted date will not be there after deleted date
How do I achieve this in pyspark?