How to generate an AWS profile in `~/.aws/config` for use in a CodeBuild Project

user_1887261 0
Viewed 827

I'm working with a tool called dbt and the database authentication method for the tool utilizes IAM. Unfortunately, IAM profiles do not exist when CodeBuild projects are built because it utilizes Instance Profiles instead. Beacause of this I am unable to connect to my database.

Referencing this question, I tried running aws sts get-caller-identity in the project to see if I was able to get some of the values I need returned, but it returned

botocore.exceptions.ProfileNotFound: The config profile (***) could not be found

Does anyone have idea on how to generate my own ~/.aws/config within a CodeBuild project?

edit: the tool uses boto3 to generate temporary credentials here: https://github.com/fishtown-analytics/dbt/blob/9d00c000720d17c42a4fa08a26b75bd500cc857f/plugins/redshift/dbt/adapters/redshift/connections.py#L101-L123

but it does not seem to be able to generate those credentials within a CodeBuild project.

edit:

buildspec.yml

version: 0.2

env:
  variables:
    MODELS_REPO: dbt-dev
    PYTHON_VERSION: 3.8
  parameter-store:
    AWS_ENVIRONMENT: "/cloudformation/environment"
    AWS_PROFILE: "/cloudformation/environment"
    CODEARTIFACT_COMPANY: "/codeartifact/company"
    GITHUB_OWNER: "/github/owner"
    GITHUB_PERSONAL_ACCESS_TOKEN: "/secret/github/token"
    GITHUB_USER: "/github/user"

phases:
  install:
    runtime-versions:
        python: "${PYTHON_VERSION}"
    commands:
      - pip install -r projects/${PROJECT_NAME}/requirements.txt
      - ./projects/${PROJECT_NAME}/.aws/phases/install.sh
  pre_build:
    commands:
      - ./projects/${PROJECT_NAME}/.aws/phases/pre_build.sh
  build:
    commands:
      - ./projects/${PROJECT_NAME}/.aws/phases/build.sh
  post_build:
    commands:
      - ./projects/${PROJECT_NAME}/.aws/phases/post_build.sh

cache:
  paths:
    - /root/.cache/pip
    - /root/.cache/pip/**/*
    - ~/.cache/pip
    - ~/.cache/pip/**/*
dbt aws-codebuild amazon-iam amazon-web-services
1 Answers

Following script should work for your use-case:

apt install jq -y
creds=$(aws sts get-session-token)

AWS_ACCESS_KEY_ID=$(echo $creds | jq '.Credentials.AccessKeyId')
AWS_SECRET_ACCESS_KEY=$(echo $creds | jq '.Credentials.SecretAccessKey')
AWS_SESSION_TOKEN=$(echo $creds | jq '.Credentials.SessionToken')

aws configure --profile $AWS_PROFILE set region "us-east-1"
aws configure --profile $AWS_PROFILE set output "json"
aws configure --profile $AWS_PROFILE set aws_secret_access_key "$AWS_SECRET_ACCESS_KEY"
aws configure --profile $AWS_PROFILE set aws_access_key_id "$AWS_ACCESS_KEY_ID"
aws configure --profile $AWS_PROFILE set aws_session_token "$AWS_SESSION_TOKEN"

You can change the region for your needs.

user_99074810
Related