Filter ansible dict attribute values without 'selectattr'

user_19259635 0
Viewed 113

I need to parse a text file for gaps in a numbered list, then select the first missing number as an Ansible variable.

For example, with a list that looks like this:

100
101
102
103
104
106
107
109
110

My requirement is to identify the first integer missing in the sequence 100..111 from this file, in this case, it would need to select 105.

I am currently using lineinfile to parse the text:

- lineinfile:
    dest: target.txt
    regexp: "{{ item }}"
    state: absent
  check_mode: yes
  register: search
  loop: "{{ range(100, 111) | list }}"

This gives me a dictionary called search that contains, among other things, a found value for each number. I'd like to filter the dictionary to select the first number occurring with value found: '0', but have so far been unsuccessful.

Due to an outdated version of Ansible and Jinja2 (that can't be updated for reasons), I don't have access to the selectattr or rejectattr filters, which I know would be the preferred way of accomplishing this. I'm currently trying to develop a workaround using json_query but haven't had any luck yet.

I don't care how cumbersome the final code is.

ansible filter jinja2 python json
3 Answers

You could do a simple loop, with the conditions:

  • the variable is not yet defined
  • found is zero
- set_fact:
    first_missing: "{{ item.item }}"
  loop: "{{ search.results }}"
  when: first_missing is not defined and item.found == 0
  loop_control:
    label: "{{ item.item }}"

This would yield:

TASK [set_fact] **************************************************************
skipping: [localhost] => (item=100) 
skipping: [localhost] => (item=101) 
skipping: [localhost] => (item=102) 
skipping: [localhost] => (item=103) 
skipping: [localhost] => (item=104) 
ok: [localhost] => (item=105)
skipping: [localhost] => (item=106) 
skipping: [localhost] => (item=107) 
skipping: [localhost] => (item=108) 
skipping: [localhost] => (item=109) 
skipping: [localhost] => (item=110)

And you effectively ends with 105 inside the variable first_missing.

But your idea to have a JMESPath query is pretty clever, and can do it too, the set_fact becomes as simple as:

- set_fact:
    first_missing: >-
      {{
        search.results
        | json_query('[?found == `0`].item | [0]')
      }}

Where

  • the expression in the square brackets ?found == `0` allows you to select all the elements of the list having found equal to zero
  • .item only select the item attribute of the dictionaries in the list
  • and the last bit resets the projection to select the first item of the list | [0]

A third solution, which is faster, is to use the diff mode along with the check_mode on a copy module, recreating the file without the gaps:

- copy:
    content: >-
      {% for i in range(100, 111) -%}
        {{ i }}
      {% endfor -%}
    dest: target.txt
  diff: yes
  check_mode: yes
  register: file

This will yield:

TASK [copy] *******************************************************************
--- before: target.txt
+++ after: /root/.ansible/tmp/ansible-local-36701z60diga/tmp2ao9r38b
@@ -3,7 +3,9 @@
 102
 103
 104
+105
 106
 107
+108
 109
 110

changed: [localhost]

Sadly, the registered result is not as smooth as the output and gives one big before/after state of the file, so you fall back to a solution similar to the one proposed by @Zeitounator:

- set_fact:
    first_missing: >-
      file.diff.0.after.split()
      | difference(file.diff.0.before.split())
      | first
user_21235300

You can do this a much simpler way IMO with a very simple set of filters:

  1. read your entries from your file with the file lookup. If the file is remote, fetch it first from the target or slurp its content into a var
  2. split the content from the file on the new line char to obtain a list
  3. map the int function to each list element to transform the parsed strings to integers
  4. apply the difference filter against the range of numbers you are looking for
  5. keep the fist element of the result list (after sorting for security)

Simply put as code taking for granted your file is locally available in target.txt

  - name: find first missing number
    debug:
      msg: "{{ range(100, 111) | list | difference(lookup('file', 'target.txt').split('\n') | map('int')) | sort | first }}"

If your original list can contain values outside of the given explored range, you can simply reject the values out-of-range prior to selecting the first result

    - name: find first missing number
      vars:
        first: 100
        last: 110
        my_range: "{{ range(first, last+1) | list }}"
        my_numbers: "{{ lookup('file', 'target.txt').split('\n') | map('int') }}"
        my_diff: "{{ my_range | difference(my_numbers) }}"
        my_diff_in_range: "{{ my_diff | reject('<', first) | reject('>', last) }}"
        my_result: "{{ my_diff_in_range | sort | first }}"
      debug:
        var: my_result
user_94010960

Create a filter, for example

shell> cat filter_plugins/range_gaps.py
def range_gaps(a):
    gaps = []
    for i in range(len(a) - 1):
        if a[i + 1] - a[i] > 1:
            gaps.extend(range(a[i] + 1, a[i + 1]))
    return (gaps)


class FilterModule(object):

    def filters(self):
        return {
            'range_gaps': range_gaps,
        }

Test it. The task below

    - debug:
        msg: "{{ item|range_gaps|to_json }}"
      loop:
        - []
        - [100]
        - [100, 101]
        - [100, 104]
        - "{{ lookup('file', 'file.txt').splitlines()|map('int')|sort }}"

gives (abridged)

  msg: '[]'
  msg: '[]'
  msg: '[]'
  msg: '[101, 102, 103]'
  msg: '[105, 108]'
user_64825610
Related