I need a Gitlab CI setup for integration tests with a mysql database (plain mysql image) and a dedicated flyway migration (custom docker image). Therefore I tried to define mysql as a service and my custom docker image as a service. For some reason flyway is unable to connect to the MySQL database (maybe because it is started before the MySQL db is ready for connections).
I am migrating from Travis CI to the free Gitlab CI version (shared runner). In Travis CI I used a docker-compose file to run the db, waited for it to be reachable, then ran the flyway migrations and after that my integration tests connected to the MySQL db.
Please note, that the flyway migrations are in a dedicated docker image for a reason. I know that my issue would be solvable by merging flyway into the mysql image or merging it into the application itself. However there are technical reasons in my use case not to do so.
Alternative steps I tried so far:
Spinning up mysql db and flyway manually with docker:dind. While this works, I am unable to connect to the mysql db from my integration tests. This seems to be a limitation of docker in docker.
Running the flyway migrations manually with docker:dind against a mysql service. In this case flyway cannot connect to the mysql service as they are in different networks.
I have created a micro example where I am trying to run a simple flyway migration against a plain MySQL database. It's a bit recursive the docker image needs to be build once first. You can see the code here: https://gitlab.com/Brutus5000/gitlab-services-test.
The gitlab-ci.yml is the following:
image: docker:stable
services:
- docker:dind
- mysql:5.7
- name: registry.gitlab.com/brutus5000/gitlab-services-test:dev
command: ["migrate"]
variables:
CONTAINER_IMAGE: registry.gitlab.com/brutus5000/gitlab-services-test
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
MYSQL_DATABASE: test
MYSQL_ROOT_PASSWORD: test
FLYWAY_URL: jdbc:mysql://mysql/test?useSSL=false
FLYWAY_USER: root
FLYWAY_PASSWORD: test
stages:
- build
build:
stage: build
script:
- docker build --tag $CONTAINER_IMAGE:dev .
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.gitlab.com
- docker push $CONTAINER_IMAGE:dev
The dockerfile is a simple addition to the base flyway image:
FROM boxfuse/flyway:5.2.0-alpine
ENV FLYWAY_EDITION community
COPY migrations /flyway/sql/
I would expect to see the flyway migrations happen in the pipeline log. Instead I see this error:
https://gitlab.com/Brutus5000/gitlab-services-test/-/jobs/139577852
*** WARNING: Service runner-72989761-project-10117031-concurrent-0-registry.gitlab.com__brutus5000__gitlab-services-test-2 probably didn't start properly.
Health check error: exit code 1
Health check container logs: 2018-12-28T12:35:50.592931651Z No HOST or PORT
Service container logs: 2018-12-28T12:35:49.751790056Z Flyway Community Edition 5.2.0 by Boxfuse 2018-12-28T12:35:50.942865457Z ERROR: 2018-12-28T12:35:50.942913738Z Unable to obtain connection from database (jdbc:mysql://mysql/test?useSSL=false) for user 'root': Communications link failure 2018-12-28T12:35:50.942919338Z 2018-12-28T12:35:50.942922129Z The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. 2018-12-28T12:35:50.942925648Z --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2018-12-28T12:35:50.942929549Z SQL State : 08S01 2018-12-28T12:35:50.942932188Z Error Code : 0 2018-12-28T12:35:50.942934751Z Message : Communications link failure 2018-12-28T12:35:50.942937423Z 2018-12-28T12:35:50.942939881Z The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. 2018-12-28T12:35:50.942943453Z