AWS Glue Error - An error occurred (403) when calling the HeadObject operation: Forbidden

user_10133469 0
Viewed 1254

I have a simple Glue pythonshell job and for testing purpose I just have print("Hello World") in it.

I have given it the required AWSGlueServiceRole. When I am trying to run the job it throws the following error:

Traceback (most recent call last):
  File "/tmp/runscript.py", line 114, in <module>
    temp_file_path = download_user_script(args.scriptLocation)
  File "/tmp/runscript.py", line 91, in download_user_script
    download_from_s3(args.scriptLocation, temp_file_path)
  File "/tmp/runscript.py", line 81, in download_from_s3
    s3.download_file(bucket_name, s3_key, new_file_path)
  File "/usr/local/lib/python3.6/site-packages/boto3/s3/inject.py", line 172, in download_file
    extra_args=ExtraArgs, callback=Callback)
  File "/usr/local/lib/python3.6/site-packages/boto3/s3/transfer.py", line 307, in download_file
    future.result()
  File "/usr/local/lib/python3.6/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.6/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/s3transfer/download.py", line 345, in _submit
    **transfer_future.meta.call_args.extra_args
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

When I add S3 full access policy to the role, then the job runs successfully. I am not able to debug what is wrong

aws-glue amazon-web-services amazon-s3
1 Answers

In Glue you need to attach S3 policies to the Amazon Glue Role that you are using to run the job. When you define the job you select the role. In this example it is AWSGlueServiceRole-S3IAMRole. That does not have S3 access until you assign it.

enter image description here

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "*"
        }
    ]
}
user_9099460
Related