Slow Resilio Sync between two Docker containers on same host

user_614770 0
Viewed 695

I'm testing out Resilio (formerly BitTorrent) Sync using two docker containers on the same host, and I've noticed that syncing files is very slow when they are added via bind mounts.

First Resilio container:

docker run  -d --name resilio1 -p 81:8888 -p 55555:55555 -v /c/Users/Test/resilio/resilio1/config:/mnt/sync -v /c/Users/Test/resilio/resilio1/data:/mnt/mounted_folders/data --restart on-failure resilio/sync

Second Resilio container (I've updated the configuration to listen on port 55556 to avoid a conflict with resilio1):

docker run  -d --name resilio2 -p 82:8888 -p 55556:55556 -v /c/Users/Test/resilio/resilio2/config:/mnt/sync -v /c/Users/Test/resilio/resilio2/data:/mnt/mounted_folders/data --restart on-failure resilio/sync

It takes about 10 minutes for even small files to sync when they are added to the host's directories that are bind mounted into the containers.

Conversely, if I docker exec into the containers and create files or folders the syncing happens immediately.

Is there something I'm missing here?

bittorrent-sync docker
1 Answers

Resilio Sync uses two mechanisms to track changes in a file system:

  • File system notifications Resilio Sync subscribes to (to obtain info about changes in a file system immediately). These notifications are just an OS kernel mechanism, for example Inotify API is used on Linux.
  • Periodic re-scan of all folders added in Sync (occurs every 10 mins by default and can be configured in settings)

As you may know Docker on Windows works using a Linux VM and relies on SMB/CIFS support in Linux kernel. As Jochem Kuijpers mentioned the issue is related to a file system notifications which don't work properly in case of Windows and mounted volumes and this is a known issue.

The workaround is to use some 3rd party tool that tracks changes in mounted windows volumes and notifies Docker containers about it.

You can use docker-windows-volume-watcher for example. Install is pretty simple (the script needs to be installed on Windows):

pip install docker-windows-volume-watcher

Usage is simple too:

docker-volume-watcher <container_name> C:\path\to\mounted\directory

As the article says:

The script will inspect all running containers and start notifying containers about changes in mounted directories. The script will also listen container start/stop events and update the list of watched directories.

user_63565380
Related