boto3 can not find the config profile

user_702846 0
Viewed 4541

my boto3 setup works with providing the credentials through /.aws/credentials - how ever I would like to pass that as environmental variable to work through docker.

this is the content of /.aws/credentials

[default]
aws_access_key_id = ABCD123
aws_secret_access_key = BCDSA123

[testing]
source_profile = default
role_arn = arn:aws:iam::123:role/access-db

everything works - but if I make a setup env variable in a aws.env file

export AWS_ACCESS_KEY_ID="ABCD123"
export AWS_SECRET_ACCESS_KEY="BCDSA123"
export AWS_ROLE_SESSION_NAME="default"
export AWS_PROFILE="testing"
export AWS_DEFAULT_PROFILE="testing"
export AWS_ROLE_ARN="arn:aws:iam::123:role/access-db"

I get the following error

root@64813e0cc755:/rate_prediction# python test.py 
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    boto3.setup_default_session(profile_name = 'testing')
  File "/usr/local/lib/python3.7/site-packages/boto3/__init__.py", line 34, in setup_default_session
    DEFAULT_SESSION = Session(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/boto3/session.py", line 80, in __init__
    self._setup_loader()
  File "/usr/local/lib/python3.7/site-packages/boto3/session.py", line 120, in _setup_loader
    self._loader = self._session.get_component('data_loader')
  File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 685, in get_component
    return self._components.get_component(name)
  File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 924, in get_component
    self._components[name] = factory()
  File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 158, in <lambda>
    lambda:  create_loader(self.get_config_variable('data_path')))
  File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 241, in get_config_variable
    logical_name)
  File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 293, in get_config_variable
    return provider.provide()
  File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 390, in provide
    value = provider.provide()
  File "/usr/local/lib/python3.7/site-packages/botocore/configprovider.py", line 451, in provide
    scoped_config = self._session.get_scoped_config()
  File "/usr/local/lib/python3.7/site-packages/botocore/session.py", line 340, in get_scoped_config
    raise ProfileNotFound(profile=profile_name)
botocore.exceptions.ProfileNotFound: The config profile (testing) could not be found

and here is my python script

import boto3
import os

boto3.setup_default_session(profile_name = 'testing')
s3 = boto3.resource('s3')
s3_client = boto3.client('s3')
my_bucket = s3.Bucket('ds-models-testing')

which I am expecting to work with - again, seem there is sth missing from my environmental variable and boto is searching for a config file botocore.exceptions.ProfileNotFound: The config profile (testing) could not be foun

boto boto3 amazon-web-services
3 Answers

You need to unset the AWS_PROFILE env var.

user_3907380

You don't have config file. config and credentials file should be in ~/.aws folder.

.aws/credentials

[default]
aws_access_key_id = ABCD123
aws_secret_access_key = BCDSA123

and in config file .aws/config

[profile testing]
source_profile = default
role_arn = arn:aws:iam::123:role/access-db
user_32305890

First, I think you should remove this line if you are trying to read from environment variables:

boto3.setup_default_session(profile_name = 'testing')

Documentation says

The order in which Boto3 searches for credentials is:

1. Passing credentials as parameters in the boto.client() method
2. Passing credentials as parameters when creating a Session object
3. Environment variables
4. Shared credential file (~/.aws/credentials)
5. AWS config file (~/.aws/config)
6. Assume Role provider
7. Boto2 config file (/etc/boto.cfg and ~/.boto)
8. Instance metadata service on an Amazon EC2 instance that has an IAM role configured.

Reference: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

So as you suspect, most likely the environment variables are not getting set as you expect. Try adding code to log the value of an environment variable that you are setting via the app.env file just to be sure you actually see it set and can read it as an environment variable:

os.environ.get('AWS_ACCESS_KEY_ID', 'It is not set!')

Assuming it is actually reading the environment variables you are setting in aws.env, then try exporting only these 2 environment variables in your aws.env and removing the others:

export AWS_ACCESS_KEY_ID="ABCD123"
export AWS_SECRET_ACCESS_KEY="BCDSA123"
user_2300550
Related