Liquibase via Docker - Changelog is not written to disk

Question

I want to set up Liquibase (using Docker) for a PostgreSQL database running locally (not in a container). I followed multiple tutorials, including the one on Docker Hub.

As suggested I've created a liquibase.docker.properties file in my

classpath: /liquibase/changelog
url: jdbc:postgresql://localhost:5432/mydb?currentSchema=public
changeLogFile: changelog.xml
username: myuser
password: mypass

to be able to run docker run --rm --net="host" -v :/liquibase/changelog liquibase/liquibase --defaultsFile=/liquibase/changelog/liquibase.docker.properties .

When I run [...] generateChangeLog I get the following output (with option --logLevel info):

[2021-04-27 06:08:20] INFO [liquibase.integration] No Liquibase Pro license key supplied. Please set liquibaseProLicenseKey on command line or in liquibase.properties to use Liquibase Pro features.
Liquibase Community 4.3.3 by Datical
####################################################
##   _     _             _ _                      ##
##  | |   (_)           (_) |                     ##
##  | |    _  __ _ _   _ _| |__   __ _ ___  ___   ##
##  | |   | |/ _` | | | | | '_ \ / _` / __|/ _ \  ##
##  | |___| | (_| | |_| | | |_) | (_| \__ \  __/  ##
##  \_____/_|\__, |\__,_|_|_.__/ \__,_|___/\___|  ##
##              | |                               ##
##              |_|                               ##
##                                                ## 
##  Get documentation at docs.liquibase.com       ##
##  Get certified courses at learn.liquibase.com  ## 
##  Free schema change activity reports at        ##
##      https://hub.liquibase.com                 ##
##                                                ##
####################################################
Starting Liquibase at 06:08:20 (version 4.3.3 #52 built at 2021-04-12 17:08+0000)
BEST PRACTICE: The changelog generated by diffChangeLog/generateChangeLog should be inspected for correctness and completeness before being deployed.
[2021-04-27 06:08:22] INFO [liquibase.diff] changeSets count: 1
[2021-04-27 06:08:22] INFO [liquibase.diff] changelog.xml does not exist, creating and adding 1 changesets.
Liquibase command 'generateChangeLog' was executed successfully.

It looks like the command ran "successfully" but I could not find the file changelog.xml in my local directory which I mounted, i.e. . The mounting however has to be working since it connects to the database successfully, i.e. the container is able to access and read liquibase.docker.properties.

First I thought I might have to "say" to Docker that it is allowed to write on my disk but it seems that this should be supported [from the description on Docker Hub]:

The /liquibase/changelog volume can also be used for commands that write output, such as generateChangeLog

What am I missing? Thanks in advance for any help!

Additional information

Output of docker inspect:

"Mounts": [
    {
        "Type": "bind",
        "Source": "",
        "Destination": "/liquibase/changelog",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
    },
    ...
],

Nathan_Voxland_45756 · Answer

When you run generateChangeLog, the path to the file should be specified as /liquibase/changelog/changelog.xml even though for update it needs to be changelog.xml

Example:

docker run --rm --net="host" -v :/liquibase/changelog liquibase/liquibase --defaultsFile=/liquibase/changelog/liquibase.docker.properties --changeLogFile=/liquibase/changelog/changelog.xml generateChangeLog

For generateChangeLog, the changeLogFile argument is the specific path to the file to output vs. a path relative to the classpath setting that update and other commands use.

When you include the command line argument as well as a defaultsFile like above, the command line argument wins. That lets you leverage the same default settings while replacing specific settings when specific commands need more/different ones.

Details

There is a distinction between operations that are creating files and ones that are reading existing files.

With Liquibase, you almost always want to use paths to files that are relative to directories in the classpath like the examples have. The specified changelogFile gets stored in the tracking system, so if you ever run the same changelog but referenced in a different way (because you moved the root directory or are running from a different machine) then Liquibase will see it as a new file and attempt to re-run already ran changesets.

That is why the documentation has classpath: /liquibase/changelog and changeLogFile: com/example/changelog.xml. The update operation looks in the /liquibase/changelog dir to find a file called com/example/changelog.xml and finds it and stores the path as com/example/changelog.xml.

GenerateChangeLog is one of those "not always relative to classpath" cases because it needs to know where to store the file. If you just specify the output changeLogFile as changelog.xml it creates just creates that file relative to your process's working directory which is not what you are needing/expecting.

user_9478384 · Answer

TL;DR

Prefix the changelog filename with /liquibase/changelog/ and pass it as a command line argument:

[...] --changeLogFile /liquibase/changelog/changelog.xml generateChangelog

See Nathan's answer for details.

Explanation

I launched the container with -it and overwrote the entrypoint to get an interactive shell within the container (see this post):

docker run --net="host" -v :/liquibase/changelog -it --entrypoint /bin/bash liquibase/liquibase -s

Executing ls yields the following:

liquibase@ubuntu-rafael:/liquibase$ ls
ABOUT.txt            UNINSTALL.txt  docker-entrypoint.sh  liquibase
GETTING_STARTED.txt  changelog      examples              liquibase.bat
LICENSE.txt          changelog.txt  lib                   liquibase.docker.properties
README.txt           classpath      licenses              liquibase.jar

Notable here is docker-entrypoint.sh which actually executes the liquibase command, and the folder changelog which is mounted to my local (my .properties file is in there).

Now I ran the same command as before but now inside the container:

sh docker-entrypoint.sh --defaultsFile=/liquibase/changelog/liquibase.docker.properties --logLevel info generateChangeLog

I got the same output as above but guess what reveals when running ls again:

ABOUT.txt            changelog             examples       liquibase.docker.properties
GETTING_STARTED.txt  changelog.txt         lib            liquibase.jar
LICENSE.txt          changelog.xml         ...

The changelog actually exists! But it is created in the wrong directory...

If you prefix the changelog filename with /liquibase/changelog/, the container is able to write it to your local (mounted) disk.

P.S. This means that the description of the "Complete Example" using "a properties file" from here is not working. I will open an Issue for that.

UPDATE

Specifying the absolute path is only necessary for commands that write a new file, e.g. generateChangeLog (see Nathan's answer). But it is better practise to pass the absolute path via command line so that you can keep the settings in the defaults-file.

Liquibase via Docker - Changelog is not written to disk

2 Answers

TL;DR

Explanation

UPDATE