Multiple Database Setup Github Actions

Question

I have defined a secondary external database in my Rails application for read-only purposes. Thus for testing, I setup a local database and plan to mock data within my test examples. Connecting to the database and running tests locally work great. However, when running the CI tests, the secondary database fails to setup due to the following error:

I believe this to be a configuration setup issue within the ci.yml file, and am not sure how to configure this properly.

# ci.yml

name: Continuous Integration
on:
  pull_request:
    branches: [ main ]
jobs:
  test:
    name: Testing
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:12
        ports:
          - "5432:5432"
        env:
          POSTGRES_USER: rails
          POSTGRES_PASSWORD: password
    env:
      RAILS_ENV: test
      RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Set up Chromedriver
        uses: nanasess/setup-chromedriver@v1
        # with:
          # Optional: do not specify to match Chrome's version
          # chromedriver-version: '88.0.4324.96'
      # Add or replace dependency steps here
      - name: Install Ruby and gems
        uses: ruby/setup-ruby@1a68550f2e3309e13c8ccb91ac6b8786f59ee147
        with:
          bundler-cache: true
      # Add or replace database setup steps here
      - name: Set up primary database
        env:
          POSTGRES_DB: calendarize_test
          DATABASE_URL: "postgres://rails:password@localhost:5432/calendarize_test"
        run: bin/rails db:create:primary db:migrate:primary
      - name: Set up warehouse database
        env:
          POSTGRES_DB: warehouse_test
          DATABASE_URL: "postgres://rails:password@localhost:5432/warehouse_test"
        run: bin/rails db:create:warehouse db:migrate:warehouse
      # Add or replace test runners here
      - name: Start Chromedriver
        run: |
          export DISPLAY=:99
          chromedriver --url-base=/wd/hub --disable-dev-shm-usage &
          sudo Xvfb -ac :99 -screen 0 1280x1024x24 > /dev/null 2>&1 & # optional
      - name: Run tests
        run: bundle exec rspec --color

# database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

calendarize: &calendarize
  <<: *default
  host: localhost
  username: <%= ENV["CALENDARIZE_DATABASE_USERNAME"] %>
  password: <%= ENV["CALENDARIZE_DATABASE_PASSWORD"] %>

test:
  primary:
    <<: *calendarize
    database: calendarize_test
  warehouse:
    <<: *calendarize
    database: warehouse_test
    migrations_paths: db/warehouse_migrate

development:
  primary:
    <<: *calendarize
    database: calendarize_development
  warehouse:
    <<: *calendarize
    database: warehouse_development
    migrations_paths: db/warehouse_migrate

production:
  primary:
    <<: *calendarize
    database: <%= ENV["CALENDARIZE_DATABASE_NAME"] %>
  warehouse:
    <<: *default
    url: <%= ENV["WAREHOUSE_DATABASE_URL"] %>
    database_tasks: false

user_5194116 · Answer

I think i encountered the same issue when i run tests in circleci while supporting multiple database. Because circleci assumes you will only need 1 database. In my case i was using mysql but i think same approach will also help you too. First i installed the mysql client in order to make sql queries from the image that we are on in circleci. Then i gave permissions to the user that i have in my database.yml and rest is just rails commands. Hope the code below demosntrate it better.

version: 2.1
jobs:
  test:
    parallelism: 3
    docker:
      - image: cimg/ruby:3.0.3
      - image: cimg/redis:6.2.6
      - image: cimg/mysql:8.0
        environment:
          MYSQL_ROOT_PASSWORD: rootpw
          MYSQL_USER: myuser
          MYSQL_PASSWORD: wq1234
    steps:
      - checkout
      - run:
          name: Waiting for MySQL to be ready
          command: |
            for i in `seq 1 10`;
            do
              nc -z 127.0.0.1 3306 && echo Success && exit 0
              echo -n .
              sleep 1
            done
            echo Failed waiting for MySQL && exit 1
      - run: sudo apt-get update
      - run: sudo apt-get install qt5-default libqt5webkit5-dev gstreamer1.0-plugins-base gstreamer1.0-tools gstreamer1.0-x
      - run: bundle install
      - run: sudo apt-get install mysql-client
      - run: mysql -u root -prootpw -e "GRANT ALL PRIVILEGES ON *.* TO 'myuser'@'%' WITH GRANT OPTION" --protocol=tcp
      - run: rails db:create
      - run: rails db:schema:load --trace
      - run:
          name: Test
          command: rspec

user_10335217 · Answer

The problem was solved by matching the username/password fields between both ci.yml and database.yml.

# database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  host: localhost
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

test:
  primary:
    <<: *default
    database: calendarize_test
    username: postgres
    password: postgres
  warehouse:
    <<: *default
    database: warehouse_test
    username: postgres
    password: postgres
    migrations_paths: db/warehouse_migrate

development:
  primary:
    <<: *default
    database: calendarize_development
  warehouse:
    <<: *default
    database: warehouse_development
    migrations_paths: db/warehouse_migrate

production:
  primary:
    <<: *default
    database: <%= ENV["CALENDARIZE_DATABASE_NAME"] %>
    username: <%= ENV["CALENDARIZE_DATABASE_USERNAME"] %>
    password: <%= ENV["CALENDARIZE_DATABASE_PASSWORD"] %>
  warehouse:
    <<: *default
    url: <%= ENV["WAREHOUSE_DATABASE_URL"] %>
    database_tasks: false

# ci.yml

name: Continuous Integration
on:
  pull_request:
    branches: [ main ]
jobs:
  test:
    name: Testing
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres
        ports:
          - "5432:5432"
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    env:
      RAILS_ENV: test
      RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Set up Chromedriver
        uses: nanasess/setup-chromedriver@v1
        # with:
          # Optional: do not specify to match Chrome's version
          # chromedriver-version: '88.0.4324.96'
      # Add or replace dependency steps here
      - name: Install Ruby and gems
        uses: ruby/setup-ruby@1a68550f2e3309e13c8ccb91ac6b8786f59ee147
        with:
          bundler-cache: true
      # Add or replace database setup steps here
      - name: Set up primary database
        env:
          POSTGRES_DB: calendarize_test
          DATABASE_URL: "postgresql://postgres:postgres@localhost:5432/calendarize_test"
        run: bin/rails db:create:primary db:migrate:primary
      - name: Set up warehouse database
        env:
          POSTGRES_DB: warehouse_test
          DATABASE_URL: "postgresql://postgres:postgres@localhost:5432/warehouse_test"
        run: bin/rails db:create:warehouse db:migrate:warehouse
      # Add or replace test runners here
      - name: Start Chromedriver
        run: |
          export DISPLAY=:99
          chromedriver --url-base=/wd/hub --disable-dev-shm-usage &
          sudo Xvfb -ac :99 -screen 0 1280x1024x24 > /dev/null 2>&1 & # optional
      - name: Run tests
        env:
          PG_USER: postgres
          PG_PASSWORD: postgres
        run: bundle exec rspec --color

Multiple Database Setup Github Actions

2 Answers