Replace values of specific sub dicts in nested json based on condition

Question

I have a fairly nested json structure that I want to loop over and modify ID values based on a condition. Ideally the key names are not needed as they can be variable. My idea was to grab the Id value from the top level and then search for it throughout the json file and modify all the matching values as the ID value is always the same. The modification is simply adding "_001" to it. But the condition should be that it is only done in subdicts of "type": "A". See below an example json of the structure I am dealing with:

{
    "name1": "test1",
    "Id": "12345678",
    "data": [
        {
            "type": "A",
            "Id1": "12345678",
            "level2": {
                "name2": "test2",
                "Id2": "12345678",
                "level3": {
                    "name3": "test3",
                    "Id3": "12345678"
                },
                "Edit": [
                    {
                        "Id": "12345678",
                        "Bla": "XXXXXXXXXX"
                    },
                    {
                        "Id": "12345678",
                        "Bla": "XXXXXXXXX"
                    }
                ]
            }
        },
        {
            "type": "B",
            "id": "12345678",
            "data": {
                "Id": "12345678",
                "Name": "test6"
            }
        }
    ]
}

I want to loop over this json and replace the Id value in every instance inside the level1 if the "type" is equal to "A" otherwise no need to loop through that particular dict, meaning the IDs should not be modified. I would ideally check this id value at the top level and then filter by the value instead of the key, since the key could vary a bit (see at lowest level where its called "source_id"). So in the example json all Ids within the subdict with "type": "A" should be replaced, whereas the ones with "type": "B", should remain. As output I want to return the json in the same format just with the IDs changed.

The code I have so far still relies on knowledge of the key names and does not capture the Id in the "Edit" list of dicts, as I can not figure out to search for an ID value like "12345678" in a nested dict based on the mentioned type filter:

data = json.load(open("test.json"))
curr_id = data['Id']
new_id = curr_id + '_001')
for item in data['data']:
    if item['type'] != 'B':                    
        item['Id1'] = new_id
        item['level2']['Id2'] = new_id
        item['level2']['level3']['Id3'] = new_id

user_3727050 · Answer

I would go with a recursive implementation that:

Loops data['data']
For every item there, check if we need to replace IDs
Replace IDs in that item/level
Check if that level has "Edit" and replace IDs there
Move to the next level by checking if the current item has level key in its dict

The advantage of such an approach is that you can have variable number of levels (for example level4 and 5 are possible) and it will handle them properly.

The code:

import json

def replace_id_recursively(item, old_id, new_id, level=1):
    """
    This method replaces the key on the current level
    and calls itself to replace lower levels
    """
    # This is an early-return: If the current
    # type is not "A", move on (only for level 1)
    # NOTE: is important to check level first since
    # other levels do not have 'type'
    if level == 1 and item['type'] != 'A':
        return

    # Change this level only if it has the old key
    key = f"Id{level}"
    print(f"Checking level {level} and key {key}")
    if item[key] != old_id:
        return

    item[key] = new_id

    # Check if this level has "Edit" - could be
    # split in another function
    if "Edit" in item:
        print(f" - Handling Edit in level {level}")
        for edit_item in item["Edit"]:
            edit_item["Id"] = new_id

    # Finally, before moving to the next item in
    # the list, change any children
    child = f"level{level + 1}"
    if child in item:
        replace_id_recursively(item[child], old_id, new_id, level + 1)

# MAIN
data = json.load(open("/tmp/data.json"))
old_id = data['Id']
new_id = f"{old_id}_001"

# For each item, check and replace IDs recursively
for item in data['data']:
    replace_id_recursively(item, old_id, new_id, 1)

print(json.dumps(data, indent=4))

Output:

$ python3 ./tmp/so.py 
Checking level 1 and key Id1
Checking level 2 and key Id2
 - Handling Edit in level 2
Checking level 3 and key Id3
{
    "name1": "test1",
    "Id": "12345678",
    "data": [
        {
            "type": "A",
            "Id1": "12345678_001",
            "level2": {
                "name2": "test2",
                "Id2": "12345678_001",
                "level3": {
                    "name3": "test3",
                    "Id3": "12345678_001"
                },
                "Edit": [
                    {
                        "Id": "12345678_001",
                        "Bla": "XXXXXXXXXX"
                    },
                    {
                        "Id": "12345678_001",
                        "Bla": "XXXXXXXXX"
                    }
                ]
            }
        },
        {
            "type": "B",
            "id": "12345678",
            "data": {
                "Id": "12345678",
                "Name": "test6"
            }
        }
    ]
}

Notes/assumptions:

ID is replaced only if the old/current id matches what we expect
Each item might or might not have "Edit"
"Edit" is always a list
"Edit" ID key is always "Id"
The old_id in "Edit" is not checked but is easy to add if needed

user_2111778 · Answer

Here is a short solution, call it with your_dict = replace_recurse(your_dict)

def replace_recurse(it):
    if isinstance(it, list):
        return [
            replace_recurse(entry) if entry.get('type', 'A') == 'A' else entry
            for entry in it
        ]
    if isinstance(it, dict):
        return {
            k: (v + '_001' if k.startswith('Id') else replace_recurse(v))
            for k, v in it.items()
        }
    return it

This approach blindly recurses into any dict or list it finds. Except recursion stops once it sees a dict with a 'type' defined that is not 'A'. During recursion, '_001' is appended to each field that starts with Id, so Id, Id1, and Id2 here.

Replace values of specific sub dicts in nested json based on condition

2 Answers