Access defaultdict(dict) based on list containing the keys

user_15653890 0
Viewed 52

Ok so I have a project that I am working on and I cannot figure this out. I apologize if this has been asked before, I've searched and found nothing. This is my first post.

I have some pandas dataframes that I want to access based on a hash which I've setup with:

df = defaultdict(lambda: defaultdict(dict))
or
df = defaultdict(dict)

I did this so I could index like df['a']['1'][1] or df['a'][1] depending on the use case. Note that the shape of the "matrix" will not necessarily be equal. So df['a']['2'][1] may exist but not df['b']['2'][1].

TLDR

I'd like to access the df using a list like ['a', '2', 1] or ['a', 1]

What I've done:

The old way:

I used to create master lists that I would then iterate through and check. This works but I feel like it is very ugly. It is also different for the two use cases above. I am now trying to make a wrapper around the two use cases above. I would love for the wrapper to not be a big switch for the two use cases.

x_master_list = []
y_master_list = []
for x in df:
   if x not in x_master_list:
       x_master_list.append(channel)
   for y in df[x]:
       if y not in y_master_list:
           y_master_list.append(idx)

for y in y_master_list:
        for x in x_master_list:
            if x in df:
                if y in df[x]:

The newer way:

I found a link discussing using recursion to get all of the keys. It was nice because it preserved the order of the hierarchy.

def iter_leafs(d, keys=[]):
    for key, val in d.items():
        if isinstance(val, defaultdict) | isinstance(val, dict):
            yield from iter_leafs(val, keys + [key])
        else:
            yield keys + [key]

I modified the creation of my master lists to:

def create_master_lists(type, df):
    check_type(type)
    lists = master_lists[type]
    key_list = list(iter_leafs(df))
    for key in key_list:
        for idx,list in enumerate(lists):
            if key[idx] not in list:
                list.append(key[idx])
    return lists

Now I want to do something like the following:

key_list = list(iter_leafs(df))
for y in y_master_list:
        valid_idx_keys = [key for key in keylist if key[-1] == y]

Here key_list looks like [['a','1',0],['a','1',1], etc] and valid_idx_keys is basically a filtered version.

I want to take each list from the valid_idx_keys and access df. I cannot figure out how to achieve this.

If I do the following it works, but again the point is to make a wrapper around the two use cases which do not have the same number of indexing arguments.

for x,y,z in valid_idx_keys:
    df[x][y][z]

Maybe something with recursion that slowly steps one layer down for each element in the sublists? I am still trying things, but I wanted to post here in case someone has a way to achieve this or a better solution to my problem.

defaultdict recursion python-3.x python
1 Answers

So I ended up with the following. It works but I am open to suggestions.

from collections import defaultdict

def search_dict(d, list):
    key = list[0]
    val = d.get(key)
    if isinstance(val, defaultdict) | isinstance(val, dict):
        yield from search_dict(val, list[1:])
    else:
        yield val

df = defaultdict(lambda: defaultdict(dict))
df['a']['1'][1] = 0
df['b']['1'][1] = 1

test_key_list = [['a', '1', 1], ['b','1',1]]

print(list(search_dict(df, test_key_list[0])))
print(list(search_dict(df, test_key_list[1])))

vals = []
for lis in test_key_list:
    print(lis)
    vals = vals + list(search_dict(df, lis))
print(vals)

df2 = defaultdict(dict)
df2['a'][1] = 0
df2['b'][1] = 1

test_key_list2 = [['a', 1], ['b',1]]

vals = []
for lis in test_key_list2:
    print(lis)
    vals = vals + list(search_dict(df2, lis))
print(vals)
user_156538900
Related