I have a csv file in AWS S3, when I execute a crawler the field with dates like 01/01/2016 04.21 is taken as a string in AWS glue.
How can I change it to date type?
I tried with the "modify schema" button in AWS Glue but it ends up having a blank field.
Convert the dynamic frame to a Pyspark dataframe and use Pyspark for everything. easier:
from pyspark.sql.functions import from_unixtime, unix_timestamp, col
df= dyf.toDF()
df = df.withColumn(col(columnname), from_unixtime(unix_timestamp(col(columnname),"dd/MM/yyyy hh.mm")))