Filter dictionary in a simple and readable manner

user_8713621 0
Viewed 90

Using python3.4 I'm trying to filter a firewall rulebase exported as JSON and converted to a dictionary. I want to filter based on user defined criteria but I can't seem to be able to do that without difficult to read constructs with a lot of loops and if statements.

The dict from my firewall entries is quite large so I've shortened them a litte.

2 examples:

entry1 = {'action': 'accept',
          'dstintf': [{'name': 'ZN_HDW', 'q_origin_key': 'ZN_HDW'}],
          'srcintf': [{'name': 'ZN_AUDIT', 'q_origin_key': 'ZN_AUDIT'}]
         }
entry2 = {'action': 'accept',
          'dstintf': [{'name': 'ZN_HDW', 'q_origin_key': 'ZN_HDW'}],
          'srcintf': [{'name': 'ZN_DMZ', 'q_origin_key': 'ZN_DMZ'},
                      {'name': 'ZN_MGMT', 'q_origin_key': 'ZN_MGMT'},
                      {'name': 'ZN_AUDIT', 'q_origin_key': 'ZN_AUDIT'}]
         }

I wanted to create a data structure to compare against the 2 examples and created to following:

filter = {'action': 'accept',
          'srcintf': [{'name': 'ZN_AUDIT', 'q_origin_key': 'ZN_AUDIT'}]
         }

After some searching on how to compare these structures I ended up with some readable code. My problem is it doesn't evaluate to true on entry2 (which has multiple source interfaces):

>>> filter.items() <= entry1.items()
True
>>> filter.items() <= entry2.items()
False

Any tips on how I should do this?

EDIT: Using the answer Eric Duminil below I'm able to create something (see below), although it's still not as readable as I want it. Any further tips?

example = entry2
# Compare entry to filter
noMatch = 0
for key in filter:
    if isinstance(example[key], list):
        # Convert list of dicts to list for easier comparing
        tmpExample = [d['name'] for d in example[key]]
        # Break if entry does not contain all criteria
        if not all(value in tmpExample for value in filter[key]):
            noMatch = 1
            print("No match on: " + str(filter[key]))
            break
    elif filter[key] != example[key]:
        # Simple string comparing
        noMatch = 1
        print("No match on: " + str(filter[key]))
        break

if noMatch == 0:
    print("Match")
else:
    print(" No match")
data-structures python-3.x dictionary python
1 Answers

Theory

The data structure inside 'dstintf' and 'srcintf' doesn't make this problem easier. A list of dicts is almost never the correct type.

Depending on your needs, you should probably convert it to a dict:

>>> data = [{'name': 'ZN_DMZ', 'q_origin_key': 'ZN_DMZ'},
...                       {'name': 'ZN_MGMT', 'q_origin_key': 'ZN_MGMT'},
...                       {'name': 'ZN_AUDIT', 'q_origin_key': 'ZN_AUDIT'}]
>>> {d['q_origin_key']:d['name'] for d in data}
{'ZN_AUDIT': 'ZN_AUDIT', 'ZN_MGMT': 'ZN_MGMT', 'ZN_DMZ': 'ZN_DMZ'}

But if the values for both name and q_origin_key are always equal, you could just use a list or a set:

>>> [d['name'] for d in data]
['ZN_DMZ', 'ZN_MGMT', 'ZN_AUDIT']
>>> {d['name'] for d in data}
set(['ZN_AUDIT', 'ZN_MGMT', 'ZN_DMZ'])

It should be easier to filter your data now, without loop or ifs.

Finally, dict.items() returns a list of tuples in an arbitrary order.

I don't think dict1.items() <= dict2.items() will help you for anything.

Example

It might be a good idea to split your code into multiple functions and work with sets:

query = {'action': 'accept',
         'srcintf': [{'name': 'ZN_AUDIT', 'q_origin_key': 'ZN_AUDIT'}]
         }

entry1 = {'action': 'accept',
          'dstintf': [{'name': 'ZN_HDW', 'q_origin_key': 'ZN_HDW'}],
          'srcintf': [{'name': 'ZN_AUDIT', 'q_origin_key': 'ZN_AUDIT'}]
          }

entry2 = {'action': 'accept',
          'dstintf': [{'name': 'ZN_HDW', 'q_origin_key': 'ZN_HDW'}],
          'srcintf': [{'name': 'ZN_DMZ', 'q_origin_key': 'ZN_DMZ'},
                      {'name': 'ZN_MGMT', 'q_origin_key': 'ZN_MGMT'},
                      {'name': 'ZN_AUDIT', 'q_origin_key': 'ZN_AUDIT'}]
          }


def compare_values(value1, value2):
    if isinstance(value1, list) and isinstance(value2, list):
        return set(d['name'] for d in value1).issubset(set(d['name'] for d in value2))
    else:
        return value1 == value2

def is_a_match(query, entry):
    do_not_match = [key for key in query if not compare_values(
        query[key], entry.get(key))]
    for key in do_not_match:
        print("%r does not match" % key)
    return len(do_not_match) == 0

print(is_a_match(query, entry1))
# True
print(is_a_match(query, entry2))
# True
print(is_a_match({'action': 'decline'}, entry2))
# 'action' does not match
# False
user_64190070
Related