The problem

Error photo

Flyway Job Error

I'm currently attempting to use flyway to perform migrations in postgres. I have a feeling the init scripts are unable to be found based on this error message. Not too sure what I'm missing as the init scripts worked when using docker containers.

Any help would be appreciated for solving this issue.

What I have tried:

1. So far I have tried playing with the connection string inside the job to see if it was a connection problem.
2. Then I tried using a config map with the init sql script inside to hopefully have the script visible for migrations. Didn't have too much luck with that approach. I have attached the example that I found for this approach. Migrations with ConfigMap for init sql scripts

I have some more research links that I found but they didn't quite fit or work with my problem and weren't as noteworthy or were outdated.

1. Database migrations in Kubernetes with flyway video
2. Continuous Database configuration with flyway

Code/Kubernetes setup

What follows is my current setup:

Side note: I am using kind to pull my docker images locally and create my clusters, I will attach my docker compose and docker files at the end.

Flyway Setup

I have Flyway as a Job in Kubernetes with this structure:

Flyway job file

apiVersion: batch/v1
kind: Job
metadata:
  name: flyway-migration-job
spec:
  template:
    spec:
      containers:
      - name: flyway
        image: flyway/flyway:6.3.1
        command: ["flyway -url=jdbc:postgresql://postgres-master:5432/wallet -user=test_user -password=testuser1234  -connectRetries=60 migrate"]
      restartPolicy: Never
  backoffLimit: 1

Postgres Setup

Postgres Deployment file:

This job is using postgres which I have created a deployment for using:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres-master
  labels:
    app: postgres-master
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres-master
  template:
    metadata:
      labels:
        app: postgres-master
    spec:
      containers:
        - name: postgres-master
          image: postgres:12
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_DB
              value: wallet
            - name: POSTGRES_USER
              value: test_user
            - name: POSTGRES_PASSWORD
              value: testuser1234

Postgres service file

I have then exposed the Postgres pod using this Postgres service:

apiVersion: v1
kind: Service
metadata:
  name: postgres-master
  labels: 
    app: postgres-master
spec: 
  type: NodePort
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
  selector: 
    app: postgres-master

Node setup

I also have a node deployment which connects to the database pod and I have confirmed that I have been able to connect to the database using hostname postgres-master(pod name), the above login details and port 5432.

Node deployment file

I've attached the node deployment file for completeness but it works.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: node-deployment
  labels:
    app: node-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: node-deployment
  template:
    metadata:
      labels:
        app: node-deployment
    spec:
      containers:
        - name: node
          image: wallet-api-20_web:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: PG_HOST
              value: "postgres-master.default"

Docker setup

Docker compose

Then I have my docker compose file which I use to build my images: Side not - All of the docker images worked together including migrations before I started adding Kubernetes.

version: '3.5'
services:
  web:
    build:
      dockerfile: ./Dockerfile
    container_name: wallet-api-web
    depends_on: 
      - flyway
    ports: 
      - "8080:8080"
  postgres-db:
    build:
      dockerfile: ./utils/db.Dockerfile
    container_name: postgres-db
    environment:
      POSTGRES_PASSWORD: testuser1234
      POSTGRES_USER: test_user
      POSTGRES_DB: wallet
    image: postgres:12
    ports:
      - "5432:5432"
    restart: always

  flyway:
    command: -url=jdbc:postgresql://db:5432/wallet -user=test_user -password=testuser1234  -connectRetries=60 migrate
    container_name: flyway-migrations
    depends_on:
      - db
    image: flyway/flyway:6.3.1
    volumes:
      - ${PWD}/migrations/sql:/flyway/sql

Postgres Dockerfile

Here is my dockerfile for postgres:

FROM postgres:13

ENV POSTGRES_DB wallet

Node dockerfile

FROM node:15
ENV NODE_ENV=production

WORKDIR /app

# Copying the package json first enables caching.
COPY ["package.json", "package-lock.json*", "./"]

RUN npm install --production

COPY . . 

RUN npm run build

ENTRYPOINT npm run start

Flyway Migration scripts

This is the only migration script I have and it was working before moving to Kubernetes.

CREATE TABLE transaction_events (
    transaction_version int,
    transaction_type VARCHAR(200),
    coins int,
    transaction_id VARCHAR(500),
    wallet_id VARCHAR(500)
)

Any help would be appreciated, I have a feeling this might be something small I am missing.

If anymore information is required please let me know.

Feel free to add any questions as well and I will do my best to provide further context.

Thank you.