AWS SDK can not read environment variables

Question

I am setting AWS_ env variables as below for Jenkins

    sudo apt-get update -y
    sudo apt-get install -y python3 python-pip python-devel
    sudo pip install awscli
    S3_LOGIN=$(aws sts assume-role --role-arn rolename --role-session-name s3_session)
    export AWS_CREDENTIAL_PROFILES_FILE=~/.aws/credentials
    export AWS_ACCESS_KEY_ID=$(echo ${S3_LOGIN}| jq --raw-output '.Credentials|"\(.AccessKeyId)"')
    export AWS_SECRET_ACCESS_KEY=$(echo ${S3_LOGIN} | jq --raw-output '.Credentials|"\(.SecretAccessKey)"')
    export AWS_SESSION_TOKEN=$(echo ${S3_LOGIN} | jq --raw-output '.Credentials|"\(.SessionToken)"')
    aws configure set default.region us-east-2
    aws configure set AWS_ACCESS_KEY_ID $AWS_ACCESS_KEY_ID
    aws configure set AWS_SECRET_ACCESS_KEY $AWS_SECRET_ACCESS_KEY

But when I try to get them from code the sdk can not read the env variables already set

 AWSCredentials evc = new EnvironmentVariableCredentialsProvider().getCredentials();
 AmazonS3Client amazonS3 = new AmazonS3Client(evc);
 amazonS3.setRegion(RegionUtils.getRegion("us-east-2"));

com.amazonaws.AmazonClientException: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))

The EnvironmentVariableCredentialsProvider in AWS SDK looks below,

public AWSCredentials getCredentials() {
        String accessKey = System.getenv(ACCESS_KEY_ENV_VAR);
        if (accessKey == null) {
            accessKey = System.getenv(ALTERNATE_ACCESS_KEY_ENV_VAR);
        }

        String secretKey = System.getenv(SECRET_KEY_ENV_VAR);
        if (secretKey == null) {
            secretKey = System.getenv(ALTERNATE_SECRET_KEY_ENV_VAR);
        }

        accessKey = StringUtils.trim(accessKey);
        secretKey = StringUtils.trim(secretKey);
        String sessionToken =
            StringUtils.trim(System.getenv(AWS_SESSION_TOKEN_ENV_VAR));

        if (StringUtils.isNullOrEmpty(accessKey)
                || StringUtils.isNullOrEmpty(secretKey)) {

            throw new AmazonClientException(
                    "Unable to load AWS credentials from environment variables " +
                    "(" + ACCESS_KEY_ENV_VAR + " (or " + ALTERNATE_ACCESS_KEY_ENV_VAR + ") and " +
                    SECRET_KEY_ENV_VAR + " (or " + ALTERNATE_SECRET_KEY_ENV_VAR + "))");
        }

        return sessionToken == null ?
                new BasicAWSCredentials(accessKey, secretKey)
                :
                new BasicSessionCredentials(accessKey, secretKey, sessionToken);
    }

EDIT: I try below approach also,

 ProfileCredentialsProvider  evc = new ProfileCredentialsProvider();
        AmazonS3Client amazonS3 = new AmazonS3Client(evc);
        amazonS3.setRegion(RegionUtils.getRegion("us-east-2"));

But even I set AWS_CREDENTIAL_PROFILES_FILE in the script because the credentials file is under ~/.aws/credentials, I still get below,

credential profiles file not found in the given path: /root/.aws/credentials

Even though the AwsProfileFileLocationProvider code says below, i am not sure why it try to look at /root/.aws/credentials

Checks the environment variable override * first, then checks the default location (~/.aws/credentials), and finally falls back to the * legacy config file (~/.aws/config) that we still support loading credentials from

user_2860032 · Answer

You should be able to change the credentials file location using the AWS_CREDENTIAL_PROFILES_FILE environment variable

user_4731259 · Answer

I am assuming you are configuring your Jenkins Job with different build steps between set credential and consume credential.
Jenkins does not share environment variable between build steps.

If you are using old-style of Jenkins job you will need to use some Plugin like envinject, or use a file to share the variables between steps. Like below (just as example).

Step 1

echo "export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}" > credential
echo "export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}" >> credential
echo "export AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}" >> credential

Step 2

source credential && ./your_command_here

But if you are suing Jenkins Pipeline, you can use env. Like below (just as example).

  pipeline {
        parameters {
            string(name: 'AWS_ACCESS_KEY_ID', defaultValue: '')
        }

        stage("set credential") {
             steps {
               tmp_AWS_ACCESS_KEY_ID =  sh (script: 'your shell script here', returnStdout: true).trim()
               env.AWS_ACCESS_KEY_ID = tmp_AWS_ACCESS_KEY_ID
              }
        }
        stage("consume credential") {
            steps {
              echo "${env.AWS_ACCESS_KEY_ID}"
            }
        }
  }

AWS SDK can not read environment variables

2 Answers