Rails 6 credentials in AWS Beanstalk: ArgumentError: key must be 16 bytes

Viewed 951

I've hit a wall. I'm deploying a Rails 6 app to AWS via Elastic Beanstalk. The deployment is done through the eb cli, and I'm using git to do it.

The error I'm hitting, no matter what I try, is: ArgumentError: key must be 16 bytes

This occurs whenever I'm trying to access the encrypted credentials, Rails.application.credentials.sendgrid[:api_key] as an example, that are set using environment keys with EDITOR="mvim -f" rails credentials:edit --environment production

Everything I've seen is using Rails 5.2, and everything seems to be stored using the master.key rather than environment specific yml files.

What I've tried:

  1. Setting RAILS_MASTER_KEY in the Environment Properties in the EB Web Console
  2. I can do eb printenv and I do indeed see this key
  3. In config/production.rb I have set the config.require_master_key = true
  4. I tried setting RAILS_PRODUCTION_KEY to the same thing as the master key, still no luck
  5. I added RAILS_ENV with production as the value as an Environment Property
  6. I added my own container ebextensions for running migrations and precompile things, and they still are erroring out the same way whenever I'm trying to retrieve credentials.

Overall, it seems like it's unable to get the master key correctly, and it's complaining about the key not being the correct 16 bytes.

When I run RAILS_ENV=production bundle exec rails c locally, it works just fine and I can get all the credentials.

Here's my .ebextensions/config file:

# Beanstalk ain't ready for Rails 6. This fix is courtesy of https://austingwalters.com/rails-6-on-elastic-beanstalk/
# Additional node 6 cleanup courtesy of https://github.com/nodesource/distributions/issues/486

commands:
  00_remove_node_6_if_present:
    command: "/bin/rm -rf /var/cache/yum && /usr/bin/yum remove -y nodejs && /bin/rm /etc/yum.repos.d/nodesource* && /usr/bin/yum clean all"
    ignoreErrors: true
  01_download_nodejs:
    command: "curl --silent --location https://rpm.nodesource.com/setup_8.x | sudo bash -"
  02_install_nodejs:
    command: "yum -y install nodejs"
  03_install_yarn:
    command: "sudo wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo && sudo yum install yarn -y"
  04_mkdir_webapp_dir:
    command: "mkdir /home/webapp"
    ignoreErrors: true
  05_chown_webapp_dir:
    command: "chown webapp:webapp /home/webapp"
    ignoreErrors: true
  06_chmod_webapp_dir:
    command: "chmod 0744 /home/webapp"
    ignoreErrors: true
  07_chmod_logs:
    command: "chown webapp:webapp -R /var/app/current/log/"
    ignoreErrors: true
  08_create_log_file:
    command: "touch /var/app/current/log/production.log"
    ignoreErrors: true
  09_chown_log_production:
    command: "chown webapp:webapp /var/app/current/log/production.log"
    ignoreErrors: true
  10_chmod_log_dir:
    command: "chmod 0664 -R /var/app/current/log/"
    ignoreErrors: true
  11_update_bundler:
    command: "gem update bundler"
    ignoreErrors: true
  12_config_for_update_nokogiri:
    command: "bundle config build.nokogiri --use-system-libraries"
  13_chown_current:
    command: "chown webapp:webapp -R /var/app/current/"
    ignoreErrors: true
  14_chmod_current:
    command: "chmod 0755 -R /var/app/current/"
    ignoreErrors: true
  15_chown_current:
    command: "chown webapp:webapp -R /var/app/ondeck/"
    ignoreErrors: true
  16_chown_current:
    command: "chmod 0644 -R /var/app/ondeck/"
    ignoreErrors: true

container_commands:

  17_install_webpack:
    command: "npm install --save-dev webpack"
  18_config_for_update_nokogiri:
    command: "bundle config build.nokogiri --use-system-libraries"
  19_precompile:
    command: "RAILS_ENV=production bundle exec rake assets:precompile"
  20_database_migration:
    leader_only: true
    command: "RAILS_ENV=production bundle exec rake db:migrate"
2 Answers

What solved it:

eb setenv RAILS_MASTER_KEY=XXXXXXXX

Even though I set this environment property in the EB web console, it for some reason wasn't taking. After setting it in the console, it would give me a successful update message.

Environment update completed successfully.

But as soon as I did $eb deploy I would see this in the events:

Environment update is starting.

Which leaves me to believe it was overwriting all my environment variables set in the console, or at least it was somehow setting a different subset of what I intended. As soon as I tried $eb setenv RAILS_MASTER_KEY=XXX the rails credentials could be found.

Had the same issue trying to get the ENV properties passed to rails console

Set them in the AWS web console, puma finds them, but rails console doesn't

This has only been a problem since I upgraded to Amazon Linux 2

My workaround is to re-create the .env from the get-config

files:
  "/home/ec2-user/railsc":
    mode: "000777"
    owner: root
    group: root
    content: |
      sudo su - -c "cd /var/app/current; /opt/elasticbeanstalk/bin/get-config --output YAML environment | sed 's/: /=/g' > .env; bundle exec rails c"
Related