Build two dictionaries based on a sorted list

user_14348705 0
Viewed 60

What I want is get two dictionaries based on a sorted list

The form of input list is (string, id). Example: [("car", 1), ("car", 2), ("car", 2), ("book", 1), ("banana", 3)]

The form of first output dictionary is (id, term_frequency). Example: {"car": [(1, 1), (2, 2)], "book": [(1, 1)], "banana": [(3, 1)]}

The form of second output dictionary is the unique term frequency. Example: {"car": 2, "book": 1, "banana": 1} as there are two unique terms of car, car 1 and car 2

However, I only finished part of the code and can't continue, I can't get the logical stright. As the comment described, I do have a problem on update the dictionary. It seems that I should keep iterating through the list and check the conditions, maybe an if loop?

I wonder how should I keep working from my current code.

input_list = [("car", 1), ("car", 2), ("car", 2), ("book", 1), ("banana", 3)]
def get_index(list)
    index = {}
    doc_freq = {}
    
    index = 0
    
    while(index < len(list)):
        (term, doc_id) = list[index]
        index[term] = [(doc_id, 1)]
    #here is where I can't continue
    
    for(term, value) in index.items():
        doc_freq[term] = len(value)
        

    return index, doc_freq
python
3 Answers

Taking advantage of the data already being sorted:

from itertools import groupby
from operator import itemgetter as ig

input_list = [("car", 1), ("car", 2), ("car", 2), ("book", 1), ("banana", 3)]

d1 = {k: [(v, len(vs))
          for v, [*vs] in groupby(map(ig(1), g))]
      for k, g in groupby(input_list, ig(0))}

d2 = {k: len(v) for k, v in d1.items()}
user_126710570

collections.Counter is the easiest way to count up occurrences of a given thing in a list. For example:

>>> item_ids = [("car", 1), ("car", 2), ("car", 2), ("book", 1), ("banana", 3)]
>>> from collections import Counter, defaultdict
>>> counter = Counter(item_ids)
>>> term_freq = defaultdict(list)
>>> for (item, num), count in counter.items():
...     term_freq[item].append((num, count))
...
>>> term_freq
{'car': [(1, 1), (2, 2)], 'book': [(1, 1)], 'banana': [(3, 1)]}
>>> unique_terms = dict(Counter(item for item, num in counter))
>>> unique_terms
{'car': 2, 'book': 1, 'banana': 1}
user_37997590

A slightly different format to having a tuple of (id, frequency), however you can use default dicts to get a nice clean result:

from collections import defaultdict

l =  [("car", 1), ("car", 2), ("car", 2), ("book", 1), ("banana", 3)]

def get_index(input_list):
    result = defaultdict(lambda: defaultdict(int))
    for k, v in l:
        result[k][v] += 1
    return {k: dict(v) for k, v in result.items()}  # Converts the default dict back to a normal dict, if required.

print(get_index(l))

output:
{'car': {1: 1, 2: 2}, 'book': {1: 1}, 'banana': {3: 1}}

However this will also let you look up ids faster as you can do:

result[car][car_id]
user_147203800
Related