We have an ansible to manage our Prometheus server, We use the copy module to copy the alert templates to the destination and check their syntax using promtool command:
- name: copy custom alerting rule files
copy:
src: "{{ item }}"
dest: "{{ prometheus_configuration_directory }}/rules/"
owner: root
group: '{{prometheus_group}}'
mode: '0640'
validate: "{{ prometheus_binary_directory }}/promtool check rules %s"
with_fileglob: "rules/*.yml"
notify:
- reload prometheus
But what if we want to remove an alert file from the Prometheus server? We end up with the synchronize module of ansible to sync our file with the Prometheus server, but now we cannot use validate to validate the rules, is there any other way to handle this task?