I am working with python/pySpark in Jupyter Notebook and I am trying to figure out the following:
I've got a dataframe like
MainDate Date1 Date2 Date3 Date4
2015-10-25 2015-09-25 2015-10-25 2015-11-25 2015-12-25
2012-07-16 2012-04-16 2012-05-16 2012-06-16 2012-07-16
2005-03-14 2005-07-14 2005-08-14 2005-09-14 2005-10-14
I need to compare MainDate with each of the Date1-Date4 and if MainDate == Date# then to create a new column REAL = Date#, if there is no match then REAL = "None", all the dates are in Date format, also the real dataframe has Date1 to Date72 and there could be only one match, if there is any
Final result:
MainDate Date1 Date2 Date3 Date4 REAL
2015-10-25 2015-09-25 2015-10-25 2015-11-25 2015-12-25 Date2
2012-07-16 2012-04-16 2012-05-16 2012-06-16 2012-07-16 Date4
2005-03-14 2005-07-14 2005-08-14 2005-09-14 2005-10-14 None
Thanks in advance