Invalid argument(s) 'pool_pre_ping' sent to create_engine() when running airflow initdb

Viewed 781

I have a docker-compose.yml with:

version: "3"
services:
  postgres:
    image: "postgres:9.6"
    container_name: "postgres"
    environment:
      - POSTGRES_USER=airflow
      - POSTGRES_PASSWORD=airflow
      - POSTGRES_DB=airflow
    ports:
      - "5432:5432"
    volumes:
      - ./data/postgres:/var/lib/postgresql/data  comment initdb after you will have use it at first run
  initdb:
    build: .
    entrypoint: airflow initdb
    depends_on:
      - postgres

  webserver:
    container_name: my-container
    build: .
    restart: always
    depends_on:
      - postgres
    volumes:
      - ./dags:/usr/local/airflow/dags
    ports:
      - "8080:8080"
    entrypoint: airflow webserver
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

  scheduler:
    build: .
    restart: always
    depends_on:
      - postgres
    volumes:
      - ./dags:/usr/local/airflow/dags
    entrypoint: airflow scheduler
    healthcheck:
      test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-scheduler.pid ]"]
      interval: 30s
      timeout: 30s
      retries: 3

A dockerfile with just:

FROM puckel/docker-airflow:1.10.6
RUN pip install --user psycopg2-binary
ENV AIRFLOW_HOME=/usr/local/airflow
COPY ./config/airflow.cfg /usr/local/airflow/airflow.cfg

The cfg file is the one that you cand find here, but I also tried with this one.

I run in different terminals and as a sequence:

docker-compose up postgres
docker-compose up initdb

The first one work well. But the second one returns:

initdb_1     | /usr/local/lib/python2.7/dist-packages/airflow/configuration.py:241: FutureWarning: The task_runner setting in [core] has the old default value of u'BashTaskRunner'. This value has been changed to u'StandardTaskRunner' in the running config, but please update your config before Apache Airflow 2.0.
initdb_1     |   FutureWarning
initdb_1     | [2020-03-12 16:10:07,456] {settings.py:253} INFO - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=1
initdb_1     | Traceback (most recent call last):
initdb_1     |   File "/usr/local/bin/airflow", line 25, in <module>
initdb_1     |     from airflow.configuration import conf
initdb_1     |   File "/usr/local/lib/python2.7/dist-packages/airflow/__init__.py", line 47, in <module>
initdb_1     |     settings.initialize()
initdb_1     |   File "/usr/local/lib/python2.7/dist-packages/airflow/settings.py", line 377, in initialize
initdb_1     |     configure_orm()
initdb_1     |   File "/usr/local/lib/python2.7/dist-packages/airflow/settings.py", line 266, in configure_orm
initdb_1     |     engine = create_engine(SQL_ALCHEMY_CONN, **engine_args)
initdb_1     |   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/__init__.py", line 387, in create_engine
initdb_1     |     return strategy.create(*args, **kwargs)
initdb_1     |   File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 160, in create
initdb_1     |     engineclass.__name__))
initdb_1     | TypeError: Invalid argument(s) 'pool_pre_ping' sent to create_engine(), using configuration PGDialect_psycopg2/QueuePool/Engine.  Please check that the keyword arguments are appropriate for this combination of components.

This is the minimal expression of the docer that I am trying to do. I added parameters to cfg to fix it, but nothing works.

Any idea?

0 Answers
Related