Ansible: how to achieve idempotence with tasks that append files on host (w/o reverting to initial state)

user_4498280 0
Viewed 288

I am having a hard time getting to know how to create Ansible roles that are following the best practices according to documentation. The following use-case which I am looking at is e.g. enabling Filebeat on host. Filebeat can be configured by placing a module definition in /etc/filebeat/modules.d folder.

It works fine when I am adding modules. Idempotence is working, everytime, on each run of the role (playbook), a given set of modules is enabled.

But what I should do when I decide that a given module is not longer needed? I remove it from role, rerun a playbook, so that all other modules are enabled. But: the previous run enabled a module that I am not installing directly with role after changes. So my server state is still altered in a way that is different than the role is imposing itself.

My question is: should I take care of removing modules before I apply them so I always start from, let's say, fresh state?

E.g.:

- name: Remove modules
  file:
    dest: "/etc/filebeat/modules.d/{{ item }}"
    state: absent
  loop:
    - "module1.yml"
    - "module2.yml"
    - "module3.yml" # It was being installed in previous role, but not now

- name: Enable modules via 'modules.d' directory
  template:
    src: "modules.d/{{ item }}"
    dest: "/etc/filebeat/modules.d/{{ item }}"
    mode: '0644'
  loop:
    - "module1.yml"
    - "module2.yml"

So I remove module3.yml, because I remember that I've installed it before, and install module1.yml and module2.yml.

Instead of just installing what I need, no matter what has been installed before:

- name: Enable modules via 'modules.d' directory
  template:
    src: "modules.d/{{ item }}"
    dest: "/etc/filebeat/modules.d/{{ item }}"
    mode: '0644'
  loop:
    - "module1.yml"
    - "module2.yml"

Leaving me with module1.yml and module2.yml (desired) and, unfortunately: module3.yml (from previous role).

How to manage that to avoid such situations? And avoid treating server as one big stateful machine that even if I run a role, the output is different than desired, because something has been done before that I cannot see in current Ansible role code.

Do you code revert playbooks in your Ansible workflow to revert to initial state when needed?

I am curious. Thanks in advance for your reply.

ansible-role ansible
2 Answers

In a nutshell:

- name: Configure filebeat modules
  hosts: all

  vars:
    fb_modules_d:
      - file: module1.yml
        state: present
      - file: module2.yml
        state: present
      - file: module3.yml
        state: absent

  tasks:
    - name: Make sure all needed module files are present
      template:
        src: "modules.d/{{ item.file }}"
        dest: "/etc/filebeat/modules.d/{{ item.file }}"
        mode: '0644'
      loop: "{{ fb_modules_d | selectattr('state', '==', 'present') }}"
      notifiy: restart_filebeat
 
    - name: Make sure all disabled modules are removed
      file:
        dest: "/etc/filebeat/modules.d/{{ item.file }}"
        state: "{{ item.state }}"
      loop: loop: "{{ fb_modules_d | selectattr('state', '==', 'absent') }}"
      notify: restart_filebeat

  handlers:
    - name: Restart filebeat service
      listen: restart_filebeat
      systemd:
        name: filebeat
        state: restarted

Note: I declared the variable inside the playbook for the example but that one one should most probably go inside your inventory (group or host level), and certainly not in a role (except in defaults for documentation)

user_94010960

A simpler and more robust solution is to remove all files not listed. This would be especially useful when you don't know what is the current state of the directory. For example, given the directory (relative to PWD for testing)

shell> tree etc/filebeat/modules.d
etc/filebeat/modules.d
└── module99.yml

The playbook below

shell> cat playbook.yml
- name: Configure filebeat modules
  hosts: localhost

  vars:
    fb_modules_path: etc/filebeat/modules.d
    fb_modules_d:
      - module1.yml
      - module2.yml
      - module3.yml

  tasks:
    - name: Create modules
      template:
        src: "modules.d/{{ item }}"
        dest: "{{ fb_modules_path }}/{{ item }}"
        mode: '0644'
      loop: "{{ fb_modules_d }}"
      notify: restart_filebeat

    - name: Remove modules
      file:
        dest: "{{ fb_modules_path }}/{{ item }}"
        state: absent
      loop: "{{ query('fileglob', fb_modules_path ~ '/*')|
                map('basename')|
                difference(fb_modules_d) }}"
      notify: restart_filebeat

  handlers:
    - name: Restart filebeat service
      listen: restart_filebeat
#     systemd:
#       name: filebeat
#       state: restarted
      debug:
        msg: Restart filebeat

creates the files from the list and removes others

TASK [Create modules] ***************************************************
changed: [localhost] => (item=module1.yml)
changed: [localhost] => (item=module2.yml)
changed: [localhost] => (item=module3.yml)

TASK [Remove modules] ***************************************************
changed: [localhost] => (item=module99.yml)

RUNNING HANDLER [Restart filebeat service] ******************************
ok: [localhost] => 
  msg: Restart filebeat

shell> tree etc/filebeat/modules.d
etc/filebeat/modules.d
├── module1.yml
├── module2.yml
└── module3.yml

The playbook is idempotent.

user_64825610
Related