What's an efficient way of synchronizing git pull requests and restarting systemctl services for multiple Django-Q clusters?

user_10264637 0
Viewed 29

I'm running numerous Django-Q clusters (on Ubuntu focal) to perform distributed computing on large data sets. I have production clusters and development clusters that each have their own git branch. I need a way to synchronize the updating and restarting of these cluster systemctl services when changes to their respective git repositories are made.

Currently I ssh into each machine and run the following commands.

$ git pull
$ sudo systemctl restart qcluster

I appreciate any ideas you may have.

distributed-computing django-q synchronization django git
1 Answers

The problem with this resides in its on-demand aspect.
You need to detect somehow there are new commits, and then initiate your SSH+pull+restart.

Depending on your remote hosting service (GitHub/GitLab,...), you would set a webhook listener on each reachable, to automate that step.
Once the listener receive notice of a new commit, it can in turn connect to the clusters and operate your commands.
An Ansible playbook would be a good fit, since Ansible is agentless, and relies mainly on SSH to execute scripts on remote machines.

VonC_63090
Related