How to loop over a list of dict elements deep 3 in Ansible

Viewed 522

I have the following variables in my var file:

repo_type:
  hosted:
    data:
      - name: hosted_repo1
        online: true
        storage:
          blobstarage: default
          write_policy: allow_once
      - name: hosted_repo2
        online: true
        storage:
          blobstarage: default
          write_policy: allow_once
  proxy:
    data:
      - name: proxy_repo1
        online: true
        storage:
          blobstarage: default
          write_policy: allow_once
      - name: proxy_repo2
        online: true
        storage:
          blobstarage: default
          write_policy: allow_once
  group:
    data:
      - name: group_repo1
        online: true
        storage:
          blobstarage: default
          write_policy: allow_once
      - name: group_repo2
        online: true
        storage:
          blobstarage: default
          write_policy: allow_once

I want to configure a task to to loop over (hosted,proxy and group) and body over data dict.

Here is the task:

- name: Create pypi hosted Repos
  uri:
    url: "{{ nexus_api_scheme }}://{{ nexus_api_hostname }}:{{ nexus_api_port }}\
      {{ nexus_api_context_path }}{{ nexus_rest_api_endpoint }}/repositories/pypi/{{ item.key}}"
    user: "{{ nexus_api_user }}"
    password: "{{ nexus_default_admin_password }}"
    headers:
      accept: "application/json"
      Content-Type: "application/json"
    body_format: json
    method: POST
    force_basic_auth: yes
    validate_certs: "{{ nexus_api_validate_certs }}"
    body: "{{  item  }}"
    status_code: 201
  no_log: no
  with_dict: "{{ repo_type}}"

I have tried with_items, with_dict and with_nested but nothing helped.

The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'data'

Any help would be appreciated!

2 Answers

If your goal is to loop over the contents of the data keys as a flat list, you could do it like this:

- debug:
    msg: "repo {{ item.name }} write_policy {{ item.storage.write_policy }}"
  loop_control:
    label: "{{ item.name }}"
  loop: "{{ repo_type | json_query('*.data[]') }}"

That uses a JMESPath expression to get the data key from each top-level dictionary, and then flatten the resulting nested list. In other words, it transforms you original structure into:

- name: hosted_repo1
  online: true
  storage:
    blobstarage: default
    write_policy: allow_once
- name: hosted_repo2
  online: true
  storage:
    blobstarage: default
    write_policy: allow_once
- name: proxy_repo1
  online: true
  storage:
    blobstarage: default
    write_policy: allow_once
- name: proxy_repo2
  online: true
  storage:
    blobstarage: default
    write_policy: allow_once
- name: group_repo1
  online: true
  storage:
    blobstarage: default
    write_policy: allow_once
- name: group_repo2
  online: true
  storage:
    blobstarage: default
    write_policy: allow_once

When run using your example data, this produces as output:

TASK [debug] *********************************************************************************************************
ok: [localhost] => (item=hosted_repo1) => {
    "msg": "repo hosted_repo1 write_policy allow_once"
}
ok: [localhost] => (item=hosted_repo2) => {
    "msg": "repo hosted_repo2 write_policy allow_once"
}
ok: [localhost] => (item=proxy_repo1) => {
    "msg": "repo proxy_repo1 write_policy allow_once"
}
ok: [localhost] => (item=proxy_repo2) => {
    "msg": "repo proxy_repo2 write_policy allow_once"
}
ok: [localhost] => (item=group_repo1) => {
    "msg": "repo group_repo1 write_policy allow_once"
}
ok: [localhost] => (item=group_repo2) => {
    "msg": "repo group_repo2 write_policy allow_once"
}

If you're trying to do something else, please update your question so that it clearly shows the values you expect for each iteration of your loop.

As reported by @larsk, you actually didn't manage to explain clearly how you are trying to loop over your data and what your api call is actually expecting.

But you are in luck this time since I have messed around with Nexus quite a bit (and I think I actually recognize those variable names and overall task layout)

The Nexus Repository POST /v1/repositories/pypi/[hosted|proxy|group] API endpoints are expecting one call for each of your repos in data. To implement your requirement you need to loop over the keys in repo_type to select the appropriate endpoint and then loop again over each element in data to send the repo definition to create.

This is actually possible combining a dict2items and subelements filters in your loop as in the below playbook (not directly tested).

The basics of the transformation are as follow:

  1. transform your dict to a key/value list using dict2items e.g. (shortened example)
    - key: hosted
      value:
        data:
          - <repo definition 1>
          - <repo definition 2>
    [...]
    
  2. use the subelements filter to combine each top element with each element in value.data e.g.:
    - # <- This is the entry for first repo i.e. `item` in your loop
      - key: hosted # <- this is the start of the corresponding top element i.e. `item.0` in your loop
        value:
          data:
            - <repo definition 1>
            - <repo definition 2>
      - <repo definition 1> # <- this is the sub-element i.e. `item.1` in your loop 
    - # <- this is the entry for second repo
      - key: hosted
        value:
          data:
            - <repo definition 1>
            - <repo definition 2>
      - <repo definition 2>
    [...]
    

Following one of your comments and from my experience, I added to my example an explicit json serialization of the repo definition using the to_json filter.

Putting it all together this gives:

- name: Create pypi hosted Repos
  uri:
    url: "{{ nexus_api_scheme }}://{{ nexus_api_hostname }}:{{ nexus_api_port }}\
      {{ nexus_api_context_path }}{{ nexus_rest_api_endpoint }}/repositories/pypi/{{ item.0.key }}"
    user: "{{ nexus_api_user }}"
    password: "{{ nexus_default_admin_password }}"
    headers:
      accept: "application/json"
      Content-Type: "application/json"
    body_format: json
    method: POST
    force_basic_auth: yes
    validate_certs: "{{ nexus_api_validate_certs }}"
    body: "{{ item.1 | to_json }}"
    status_code: 201
  no_log: no
  loop: "{{ repo_type | dict2items | subelements('value.data') }}"
Related