How to loop over same keys in different YAML files to generate an output per key with Ansible

user_13688126 0
Viewed 373

I have got the following YAML files:

---
U01:
  ip: 1.1.1.1
U02:
  ip: 2.2.2.2

---
U01:
  as_bgp: as1
U02:
  as_bgp: as2

I am using the following playbook to generate one output per key using the above YAML files

---
- hosts: localhost
  gather_facts: no
  
  tasks:
    - name: itterate over up nodes
      include_vars:
        dir: "vars"
        name: U

    - name: print nodes name
      template:
        src: test.j2
        dest: "outputs/{{item.key}}test.txt"
      loop: "{{ lookup('dict', U) }}"

Now, I am using the following simple Jinja2 template

{{item.value.ip}}
{{item.value.as_bgp}}

How can I modify my playbook to get the fololwing outputs (two separate files):

1.1.1.1
as1

2.2.2.2
as2

The only things that works is either using {{item.value.ip}} or {{item.value.as_bgp}} in the Jinja template, it doesn't work for both!

ansible jinja2 loops
2 Answers

If you do happen to have a really recent version of Ansible (>= 2.12), you could use the hash_behaviour parameter of the module include_vars, along with a loop and the file lookup:

- include_vars:
    name: U
    hash_behaviour: merge
    file: "{{ item }}"
  loop: "{{ lookup('fileglob', 'vars/*', wantlist=True) }}"
  vars:
    U: {}

Another option, for older version, would be to combine the two dictionaries, with the recursive=True option out of the registered output of the include_vars task:

- include_vars:
    file: "{{ item }}"
  loop: "{{ lookup('fileglob', 'vars/*', wantlist=True) }}"
  register: include

- template:
    src: test.j2
    dest: "outputs/{{item.key}}test.txt"
  loop: >-
    {{
      include.results
      | map(attribute='ansible_facts')
      | combine(recursive=True)
      | dict2items
    }}
  loop_control:
    label: "{{ item.key }}"

Given this couple of tasks:

- include_vars:
    name: U
    hash_behaviour: merge
    file: "{{ item }}"
  loop: "{{ lookup('fileglob', 'vars/*', wantlist=True) }}"
  vars:
    U: {}

- debug:
    msg: >-
      For `{{ item.key }}`,
      the IP is `{{ item.value.ip }}`
      and the BGP is `{{ item.value.as_bgp }}`
  loop: "{{ U | dict2items }}"
  loop_control:
    label: "{{ item.key }}"

Or that other couple of tasks:

- include_vars:
    file: "{{ item }}"
  loop: "{{ lookup('fileglob', 'vars/*', wantlist=True) }}"
  register: include

- debug:
    msg: >-
      For `{{ item.key }}`,
      the IP is `{{ item.value.ip }}`
      and the BGP is `{{ item.value.as_bgp }}`
  loop: >-
    {{
      include.results
      | map(attribute='ansible_facts')
      | combine(recursive=True)
      | dict2items
    }}
  loop_control:
    label: "{{ item.key }}"

They would all yield:

ok: [localhost] => (item=U01) => 
  msg: For `U01`, the IP is `1.1.1.1` and the BGP is `as1`
ok: [localhost] => (item=U02) => 
  msg: For `U02`, the IP is `2.2.2.2` and the BGP is `as2`
user_21235300

Given the tree

shell> tree vars/
vars/
├── as_bgp.yml
└── ip.yml

0 directories, 2 files

shell> cat vars/as_bgp.yml 
---
U01:
  as_bgp: as1
U02:
  as_bgp: as2

shell> cat vars/ip.yml 
---
U01:
  ip: 1.1.1.1
U02:
  ip: 2.2.2.2

The option hash_behaviour: merge works as expected (Ansible 2.12)

    - name: iterate over files
      include_vars:
        file: "{{ item }}"
        hash_behaviour: merge
        name: my_vars
      loop: "{{ query('fileglob', 'vars/*') }}"
      vars:
        my_vars: {}

gives

  my_vars:
    U01:
      as_bgp: as1
      ip: 1.1.1.1
    U02:
      as_bgp: as2
      ip: 2.2.2.2

If the parameter hash_behaviour is not available create the names of the variables from the names of the files, e.g.

    - name: iterate over files
      include_vars:
        file: "{{ item }}"
        name: "u_{{ item|basename|splitext|first }}"
      loop: "{{ query('fileglob', 'vars/*') }}"

will create the variables

  query('varnames', '^u_*'):
  - u_as_bgp
  - u_ip

In the playbook below, extract the variables, and combine the dictionary my_vars

shell> cat test.yml
- hosts: localhost
  vars:
    my_vars: "{{ query('varnames', '^u_*')|
                 map('extract', hostvars[inventory_hostname])|
                 combine(recursive=True) }}"
  tasks:
    - name: iterate over files
      include_vars:
        file: "{{ item }}"
        name: "u_{{ item|basename|splitext|first }}"
      loop: "{{ query('fileglob', 'vars/*') }}"

gives

  my_vars:
    U01:
      as_bgp: as1
      ip: 1.1.1.1
    U02:
      as_bgp: as2
      ip: 2.2.2.2

The template is trivial

shell> cat test.txt.j2
{% for k,v in my_vars.items() %}
{{ v.ip }}
{{ v.as_bgp }}

{% endfor %}

The task below

    - template:
        src: test.txt.j2
        dest: test.txt

will create the file

shell> cat test.txt
1.1.1.1
as1

2.2.2.2
as2

If the option recursive=True of the filter combine is not available merge the dictionaries on your own, e.g the variables below help to get the same result

my_vars: "{{ dict(_keys|zip(_vals)) }}"
_groups: "{{ query('varnames', '^u_*')|
             map('extract', hostvars[inventory_hostname])|
             map('dict2items')|flatten|
             groupby('key') }}"
_keys: "{{ _groups|map('first')|list }}"
_vals: "{{ _groups|map('last')|
                   map('map', attribute='value')|
                   map('combine')|list }}"
user_64825610
Related