AttributeError: 'DataFrame' object has no attribute '_mgr' on Writing a Pandas Data Frame to S3

user_15060708 0
Viewed 676

Got AttributeError: 'DataFrame' object has no attribute '_mgr' while writing a pandas data frame to S3.

import awswrangler as wr
window = '0112'
wr.s3.to_csv(
    df=mergeDf,
    path="s3://MYBUCKET/wrangled/" + "WiderData_2" +window+ ".csv"
)

Error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-65-30cb4c51bd25> in <module>
      7 # Use 'w' for py3, 'wb' for py2
      8 with s3.open('........','w') as f:
----> 9     mergeDf.to_csv(f)
.....
~/anaconda3/envs/python3/lib/python3.6/site-packages/pandas/core/generic.py in __getattr__(self, name)
   5272         """
   5273         Return a Numpy representation of the DataFrame.
-> 5274 
   5275         .. warning::
   5276 

AttributeError: 'DataFrame' object has no attribute '_mgr'

Also Tried

import s3fs
s3 = s3fs.S3FileSystem(anon=False)
# Use 'w' for py3, 'wb' for py2
with s3.open('MYBUCEKT/wrangled/WiderData_1011.csv','w') as f:
    mergeDf.to_csv(f)

Same Error 'AttributeError: 'DataFrame' object has no attribute '_mgr' '

pandas amazon-s3 python
1 Answers

just use boto3 as so

    import boto3
    import io
    import pandas as pd
    bucket='yourbucket'
    key='path/to/file.csv'
    csv_buffer = io.StringIO()
    df.to_csv(csv_buffer)
    response = boto3.client('s3').put_object(Body=csv_buffer.getvalue(), Bucket=bucket, Key=key)
user_121334340
Related