I am trying to read a large compressed CSV file from AWS S3 and convert it to a Panda data frame in Sagemaker. Is there any direct and clean approach to do it?
1 Answers
You can use the AWS Wrangler library to do so, easily
It supports GZIP compression, and will read the CSV directly into a Pandas dataframe
(pip install awswranger)
import awswrangler as wr
df = wr.s3.read_csv(path="s3://bucket/path/to/my.csv.gzip")
Related