Problem Statement: CSV data currently stored in S3 (extracted from Postgresql RDS), I need to query this S3 data using Athena. To achieve this, I created AWS Glue DB and running a crawler on this S3 bucket but the data in Athena query is broken (starts from columns with large text content). I tried changing the data type of Glue Table schema from string to varchar(1000) and recrawl, but still it breaks.
Data stored in S3 bucket :
Data coming out of Athena query on same bucket (using SELECT *) [note the missing row]
Also tested loading the S3 data using jupyter notebook in AWS Glue Studio with this code snippet and the output data looks correct here :
dynamicFrame = glueContext.create_dynamic_frame.from_options(
connection_type="s3",
connection_options={"paths": ["s3://..."]},
format="csv",
format_options={
"withHeader": True,
# "optimizePerformance": True,
},
)
Any help on this would be greatly appreciated!


