Airflow s3 fatal error: Unable to locate credentials when copying files from another company's bucket to ours

Viewed 305

I'm able to run the following command locally which is copying files from another company's bucket to our bucket:

aws s3 sync s3://client_export/other_company/pages/date={{ next_ds }}  s3://dev/our_company/pages

however when trying to run the commands in airflow locally I see the error Unable to locate credentials.

This is my task (script has the aws command I mentioned earlier):

files_to_s3 = BashOperator(
    task_id='get_files',
    bash_command=script,
    dag=dag)

files_to_s3
1 Answers
Related