Merging two data structures with Ansible

Question

Ansible v2.9.25

I'm trying to merge two data structures with Ansible. I'm almost there but I'm not able to merge all data.

Let me explain:

I want to merge main_roles together with default_roles:

main_roles:
  - name: admin
    role_ref: admin
    subjects:
      - name: test
        kind: ServiceAccount
      - name: test2
        kind: ServiceAccount
  - name: edit
    role_ref: edit
    subjects:
      - name: test
        kind: ServiceAccount


default_roles:
  - name: edit
    role_ref: edit
    subjects:
      - name: merge_me
        kind: ServiceAccount

I'm successfully combining with the following task:

- name: "Setting var roles_managed"
  set_fact:
    roles_managed: "{{ roles_managed | default([]) + [ item | combine(default_roles|selectattr('name', 'match', item.name) |list)] }}"
  loop: "{{ main_roles }}"

Printing the var via loop.

- name: "print all roles"
  ansible.builtin.debug:
       msg: "{{ item }}"
  loop: "{{ roles_managed }}"

ok: [] => (item={u'subjects': [{u'kind': u'ServiceAccount', u'name': u'test'}, {u'kind': u'ServiceAccount', u'name': u'test2'}], u'name': u'admin', u'role_ref': u'admin'}) => {
    "msg": {
        "name": "admin", 
        "role_ref": "admin", 
        "subjects": [
            {
                "kind": "ServiceAccount", 
                "name": "test"
            }, 
            {
                "kind": "ServiceAccount", 
                "name": "test2"
            }
        ]
    }
}
ok: [] => (item={u'subjects': [{u'kind': u'ServiceAccount', u'name': u'merge_me'}], u'name': u'edit', u'role_ref': u'edit'}) => {
    "msg": {
        "name": "edit", 
        "role_ref": "edit", 
        "subjects": [
            {
                "kind": "ServiceAccount", 
                "name": "merge_me"
            }
        ]
    }
}

This results in a combine on the item.name. But I want the result to be also a merge of the subjects. So i would need a end result of merge_me and test (the subjects under name:edit):

  - name: edit
    role_ref: edit
    subjects:
      - name: merge_me
        kind: ServiceAccount
      - name: test
        kind: ServiceAccount

What I'm understanding Ansible is not merging recursively by default. So I would need to set recursive=true in the combine filter. See: Combining hashes/dictionaries

But I'm not able to set this successfully in my context.

When I try: {{ roles_managed | default([]) + [ item | combine(default_role_bindings, recursive=true|selectattr('name', 'match', item.name) |list)] }} for example I'm getting an 'bool' object is not iterable" error code...

I've tried many variations and searched many other posts. But I'm still unsuccessfully after probably too many hours ;). Hoping someone has a solution!

user_6482561 · Answer

Both main_roles and default_roles are lists. It's not possible to combine lists. Instead, it's possible to add them and groupby name. Then combine the dictionaries with the same name, e.g.

    - set_fact:
        my_roles: "{{ my_roles|d([]) + [item.1|combine(list_merge='append')] }}"
      loop: "{{ (main_roles + default_roles)|groupby('name') }}"

gives

  my_roles:
  - name: admin
    role_ref: admin
    subjects:
    - kind: ServiceAccount
      name: test
    - kind: ServiceAccount
      name: test2
  - name: edit
    role_ref: edit
    subjects:
    - kind: ServiceAccount
      name: test
    - kind: ServiceAccount
      name: merge_me

Use list_merge='append', available since 2.10, to append the items of the lists.

Append the subjects on your own if the option append is not available in your version, e.g. the task below gives the same result

    - set_fact:
        my_roles: "{{ my_roles|d([]) + [item.1.0|combine({'subjects':_subj})] }}"
      loop: "{{ (main_roles + default_roles)|groupby('name') }}"
      vars:
        _subj: "{{ item.1|map(attribute='subjects')|flatten }}"

Merging two data structures with Ansible

1 Answers