How to launch a Postgresql database container alongside an Ubuntu Docker image?

Cerin 0
Viewed 27

How do you use docker-compose to launch PostgreSQL in one container, and allow it to be accessed by an Ubuntu 22 container?

My docker-compose.yml looks like:

version: "3.6"
services:
  db:
    image: postgres:14-alpine
    environment:
      - POSTGRES_USER=test
      - POSTGRES_PASSWORD=test
      - POSTGRES_DB=test
    command: -c fsync=off -c synchronous_commit=off -c full_page_writes=off --max-connections=200 --shared-buffers=4GB --work-mem=20MB
    tmpfs:
      - /var/lib/postgresql
  app_test:
    build:
      context: ..
      dockerfile: Dockerfile
      shm_size: '2gb'
    volumes:
      - /dev/shm:/dev/shm

My Dockerfile just runs a Django unittest suite that connects to the PostgreSQL database using the same credentials. However, it looks like the database periodically crashes or stops and starts, breaking connection with the tests.

When I run:

docker-compose -f docker-compose.yml -p myproject up --build --exit-code-from myproject_app

I get output like:

Successfully built 45c74650b75f
Successfully tagged myproject_app:latest
Creating myproject_app_test_1 ... done
Creating myproject_db_1       ... done
Attaching to myproject_app_test_1, myproject_db_1
db_1        | The files belonging to this database system will be owned by user "postgres".
db_1        | This user must also own the server process.
db_1        | 
db_1        | The database cluster will be initialized with locale "en_US.utf8".
db_1        | The default database encoding has accordingly been set to "UTF8".
db_1        | The default text search configuration will be set to "english".
db_1        | 
db_1        | Data page checksums are disabled.
db_1        | 
db_1        | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1        | creating subdirectories ... ok
db_1        | selecting dynamic shared memory implementation ... posix
db_1        | selecting default max_connections ... 100
db_1        | selecting default shared_buffers ... 128MB
db_1        | selecting default time zone ... UTC
db_1        | creating configuration files ... ok
app_test_1  | SITE not set. Defaulting to myproject.
app_test_1  | Initialized settings for site "myproject".
db_1        | running bootstrap script ... ok
db_1        | performing post-bootstrap initialization ... sh: locale: not found
db_1        | 2022-09-23 02:06:54.175 UTC [30] WARNING:  no usable system locales were found
db_1        | ok
db_1        | syncing data to disk ... initdb: warning: enabling "trust" authentication for local connections
db_1        | You can change this by editing pg_hba.conf or using the option -A, or
db_1        | --auth-local and --auth-host, the next time you run initdb.
db_1        | ok
db_1        | 
db_1        | 
db_1        | Success. You can now start the database server using:
db_1        | 
db_1        |     pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1        | 
db_1        | waiting for server to start....2022-09-23 02:06:56.736 UTC [36] LOG:  starting PostgreSQL 14.4 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
db_1        | 2022-09-23 02:06:56.737 UTC [36] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1        | 2022-09-23 02:06:56.743 UTC [37] LOG:  database system was shut down at 2022-09-23 02:06:56 UTC
db_1        | 2022-09-23 02:06:56.750 UTC [36] LOG:  database system is ready to accept connections
db_1        |  done
db_1        | server started
db_1        | CREATE DATABASE
db_1        | 
db_1        | 
db_1        | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1        | 
db_1        | 2022-09-23 02:06:57.100 UTC [36] LOG:  received fast shutdown request
db_1        | 2022-09-23 02:06:57.101 UTC [36] LOG:  aborting any active transactions
db_1        | 2022-09-23 02:06:57.104 UTC [36] LOG:  background worker "logical replication launcher" (PID 43) exited with exit code 1
db_1        | 2022-09-23 02:06:57.105 UTC [38] LOG:  shutting down
db_1        | waiting for server to shut down....2022-09-23 02:06:57.222 UTC [36] LOG:  database system is shut down
db_1        |  done
db_1        | server stopped
db_1        | 
db_1        | PostgreSQL init process complete; ready for start up.
db_1        | 
db_1        | 2022-09-23 02:06:57.566 UTC [1] LOG:  starting PostgreSQL 14.4 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
db_1        | 2022-09-23 02:06:57.568 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1        | 2022-09-23 02:06:57.569 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1        | 2022-09-23 02:06:57.571 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1        | 2022-09-23 02:06:57.576 UTC [50] LOG:  database system was shut down at 2022-09-23 02:06:57 UTC
db_1        | 2022-09-23 02:06:57.583 UTC [1] LOG:  database system is ready to accept connections
db_1        | 2022-09-23 02:06:58.805 UTC [57] ERROR:  relation "django_site" does not exist at character 78
db_1        | 2022-09-23 02:06:58.805 UTC [57] STATEMENT:  SELECT "django_site"."id", "django_site"."domain", "django_site"."name" FROM "django_site" WHERE "django_site"."id" = 2 LIMIT 21

Why is it reading shutting down and database system was shut down, implying the database restarts several times? Why is Django unable to access it to initialize the schema?

postgresql django docker-compose docker
1 Answers

When working with databases in docker-compose you always need to wait for them to fully start. Either your program needs to ping and wait (not crashing after the first failed attempt to connect to the database which is probably still starting up) or you can use the famous now wait-for-it.sh script.

Below is an example for the second approach.

Dockerfile:

FROM debian:stable

WORKDIR /scripts

RUN apt-get update && apt-get install -y curl telnet

# there are many versions on the internet, I just picked one
RUN curl -sO https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh && chmod a+x *.sh

ENTRYPOINT ["/scripts/wait-for-it.sh"]

it only prepares an image with the wait-for-it.sh script and telnet (to test the database connection)

docker-compose.yml

version: "3.6"

services:
  db:
    image: postgres:14-alpine
    environment:
      - POSTGRES_USER=test
      - POSTGRES_PASSWORD=test
      - POSTGRES_DB=test
    command: -c fsync=off -c synchronous_commit=off -c full_page_writes=off --max-connections=200 --shared-buffers=4GB --work-mem=20MB
    tmpfs:
      - /var/lib/postgresql

  test:
    build:
      context: .
    command: db:5432 -t 3000 -- telnet db 5432

The test service will wait for the database to be available before starting it's main process.

The best way to test it:

In one terminal start:

docker-compose up test

In a second terminal:

# make the operation even longer
docker rmi postgres:14-alpine

# start database
docker-compose up db

The reason why the database restarts during startup is well explained in the comments.

useruGkL0
Related