I have a sample data set like below
sample_data = [('A', 'Chetna', 5, 'date_add(date_format(current_date(), \'yyyy-MM-dd\'), 7)'),
('B', 'Tanmay', 6, '`date_add(date_format(current_date(), \'yyyy-MM-dd\'), 1)`'),
('C', 'CC', 2, '`date_add(date_format(current_date(), \'yyyy-MM-dd\'), 3)`'),
('D', 'TC', 9, '`date_add(date_format(current_date(), \'yyyy-MM-dd\'), 5)`')]
df = spark.createDataFrame(sample_data, ['id', 'name', 'days', 'applyMe'])
from pyspark.sql.functions import lit
df = df.withColumn("salary", lit('days * 60'))
I am trying to evaluate the function provided in applyMe column and salary.
So far have tried doing it with expr and eval but no luck.
Could someone please point me in right direction to achieve the desired output.