saving from api to s3 bucket

Question

I'm trying to get the below python code to save the csv from an api to an amazon s3 bucket using bot03 and python, but I can't see where I'm going wrong. When I execute the code I don't get any error but the file never appear in the s3 bucket.

import boto3
from botocore.exceptions import ClientError

file_name = "test.csv"
bucket = "mybucket"

def main():
    url = "https://api0.solar.sheffield.ac.uk/pvlive/v3/pes/10?start=2021-01-01T00:00:00&end=2021-07-06T00:00:00&data_format=csv"
    x = requests.get(url,headers={'Content-Type': 'application/json', 'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate',})

    s3 = bot03.client("s3")
    with open("test.csv","rb") as file2:
     s3.upload_fileobj(x.content, bucket, "test.cvc")

any tips/advice would be appreciated. I'm a python/aws newbie so apologies if a basic question

user_3946962 · Answer

I used this code to acheive what I need

file_name = "test.csv"
bucket = "my_bucket"

def main():
    url = "https://api0.solar.sheffield.ac.uk/pvlive/v3/pes/10?start=2021-01-01T00:00:00&end=2021-07-06T00:00:00&data_format=csv"
    x = requests.get(url,headers={'Content-Type': 'application/json', 'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate',})



    s3_resource = boto3.resource('s3')
    s3_resource.Object(bucket, 'snowflake/csv/df1.csv').put(Body=x.content)


if __name__ == "__main__":
    main()

saving from api to s3 bucket

1 Answers