Is there find a key deep inside a nested dictionaries and show the path to that key?
Example:
---
isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
I want to find the initial variable and print the path to it like author.fistname.initial
I would do something like this using a custom filter. E.g.:
def _findkey_helper(data, want, path=None):
if path is None:
path = []
if not data:
return False, None, None
if want in data:
return True, path + [want], data[want]
for k, v in data.items():
if isinstance(v, dict):
found, val, foundat = _findkey_helper(v, want, path=path + [k])
if found:
return (found, foundat, val)
return False, None, None
def filter_findkey(data, want):
"""Search a nested dictionary for a given key.
This filter recursively searches the nested dictionary structure `data`
for the key `want`. It returns the 3-tuple (found, path, value), where:
- `found` is True if the named key was found, False otherwise
- `path` is a dotted path to the named key, or `None` if `found` is False
- `value` is the corresponding value, or `None` if `found` is False
"""
found, path, val = _findkey_helper(data, want)
if found:
return True, ".".join(path), val
return False, None, None
class FilterModule:
def filters(self):
return {"findkey": filter_findkey}
Drop this into filter_plugins/findkey.py adjacent to your playbook, and use it like this:
- hosts: localhost
gather_facts: false
vars:
data:
isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
tasks:
- debug:
msg: "{{ data | findkey('initial') }}"
The above outputs:
TASK [debug] ********************************************************************************************
ok: [localhost] => {
"msg": "(True, 'author.firstname.initial', 'D')"
}
This only works with nested dictionary structures (that is, it won't search through lists), but necessary changes to support lists would be pretty simple.
The following playbook does the job using the to_paths lookup
---
- hosts: localhost
gather_facts: false
vars:
isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
needle: initial
tasks:
# Note: this is a necessary step to "fix" the value from the lookup.
# If you set this value in play/task/host/group variable,
# you will get a recursive loop error any time you use the resulting variable
- name: extract the info
set_fact:
matches: "{{ lookup('ansible.utils.to_paths', vars) | dict2items | selectattr('key', 'search', needle) }}"
- name: Overall info
debug:
msg: "I found {{ matches | length }} entries having '{{ needle }}' in their path"
- name: details
debug:
msg: "item at path {{ item.key }} has value: {{ item.value }}"
loop: "{{ matches }}"
Example run:
$ ansible-playbook playbook.yml
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [extract the info] ****************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Overall info] ********************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "I found 1 entries having 'initial' in their path"
}
TASK [details] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'key': 'author.firstname.initial', 'value': 'D'}) => {
"msg": "item at path author.firstname.initial has value: D"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
And to be ahead of some possible comments: this works with lists too. Replacing the var author above with:
authors:
- lastname: Doe
firstname:
name: John
initial: D
- lastname: Perez
firstname:
name: Juan
initial: J
gives with the exact same playbook:
TASK [Overall info] ********************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "I found 2 entries having 'initial' in their path"
}
TASK [details] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'key': 'authors[0].firstname.initial', 'value': 'D'}) => {
"msg": "item at path authors[0].firstname.initial has value: D"
}
ok: [localhost] => (item={'key': 'authors[1].firstname.initial', 'value': 'J'}) => {
"msg": "item at path authors[1].firstname.initial has value: J"
}
Q: "Find the initial variable and print the path to it like author.fistname.initial"
A: Use the dictionaries below. There might be more paths. For example, given the data
data:
- isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
- isbn: 123-456-333
author:
lastname: Doe
firstname:
name: Alice
initial: C
- isbn: 123-456-444
author:
lastname: Doe
firstname:
name: Bob
initial: D
- name: Create dictionary initials
set_fact:
initials: "{{ initials|d({})|combine({item: authors}) }}"
loop: "{{ data|json_query('[].*.*.initial')|flatten|unique }}"
vars:
authors: "{{ data|
selectattr('author.firstname.initial', 'eq', item)|
json_query('[].join(`,`,[author.lastname,
author.firstname.name,
author.firstname.initial])') }}"
gives
initials:
C:
- Doe,Alice,C
D:
- Doe,John,D
- Doe,Bob,D
- name: Create dictionary paths
set_fact:
paths: "{{ paths|d({})|combine(_item, list_merge='append') }}"
loop: "{{ data }}"
vars:
_paths: "{{ lookup('ansible.utils.to_paths', item) }}"
_item: "{{ dict(_paths|dict2items|json_query('[].[value, [key]]')) }}"
gives
paths:
123-456-222:
- isbn
123-456-333:
- isbn
123-456-444:
- isbn
Alice:
- author.firstname.name
Bob:
- author.firstname.name
C:
- author.firstname.initial
D:
- author.firstname.initial
- author.firstname.initial
Doe:
- author.lastname
- author.lastname
- author.lastname
John:
- author.firstname.name
paths_keys: "{{ paths.keys()|list }}"
paths_vals: "{{ paths.values()|map('unique')|list }}"
paths_unique: "{{ dict(paths_keys|zip(paths_vals)) }}"
gives
paths_unique:
123-456-222:
- isbn
123-456-333:
- isbn
123-456-444:
- isbn
Alice:
- author.firstname.name
Bob:
- author.firstname.name
C:
- author.firstname.initial
D:
- author.firstname.initial
Doe:
- author.lastname
John:
- author.firstname.name
Example of a complete playbook for testing
- hosts: localhost
vars:
data:
- isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
- isbn: 123-456-333
author:
lastname: Doe
firstname:
name: Alice
initial: C
- isbn: 123-456-444
author:
lastname: Doe
firstname:
name: Bob
initial: D
paths_keys: "{{ paths.keys()|list }}"
paths_vals: "{{ paths.values()|map('unique')|list }}"
paths_unique: "{{ dict(paths_keys|zip(paths_vals)) }}"
tasks:
- name: Create dictionary initials
set_fact:
initials: "{{ initials|d({})|combine({item: authors}) }}"
loop: "{{ data|json_query('[].*.*.initial')|flatten|unique }}"
vars:
authors: "{{ data|
selectattr('author.firstname.initial', 'eq', item)|
json_query('[].join(`,`,[author.lastname,
author.firstname.name,
author.firstname.initial])') }}"
- debug:
var: initials
- name: Create dictionary paths
set_fact:
paths: "{{ paths|d({})|combine(_item, list_merge='append') }}"
loop: "{{ data }}"
vars:
_paths: "{{ lookup('ansible.utils.to_paths', item) }}"
_item: "{{ dict(_paths|dict2items|json_query('[].[value, [key]]')) }}"
- debug:
var: paths
- debug:
var: paths_unique