Counter sorting lost when combined with defaultdict

user_4178705 0
Viewed 53

I am trying to take a list made from (item_number, fruit) tuples and count the number of times each type of fruit appears in the list. That's easy enough with collections.Counter. I'm using most_common() along with that.

The problem I'm running into is when trying to also show the list of item_numbers that correspond to a particular type of fruit they become out of order.

Here is my sample code:

#!/usr/bin/env python

from collections import Counter, defaultdict

mylist = [
            (1, 'peach'),
            (2, 'apple'),
            (3, 'orange'),
            (4, 'apple'),
            (5, 'banana'),
            (6, 'apple'),
            (7, 'orange'),
            (8, 'peach'),
            (9, 'apple'),
            (10, 'orange'),
            (11, 'plum'),
            ]

# FIRST, HANDLE JUST COUNTING THE ITEMS

normal_list = []

# append to a simple list
for item_number, fruit in mylist:
    normal_list.append(fruit)

# prints just the name of each fruit and how many times it appears
for fruit, count in Counter(normal_list).most_common(10):
    print(f'{fruit}\tCount: {count}')  

# NOW TRY TO INCLUDE THE LIST IF ITEM NUMBERS ALSO

mydefaultdict = defaultdict(list)

# append to the defaultdict
for item_number, fruit in mylist:
    mydefaultdict[fruit].append(item_number)

# prints each fruit, followed by count, and finally the list of IPs for each
for fruit, item_list in Counter(mydefaultdict).most_common(10):
    print(f'{fruit}\tCount: {len(item_list)}\tList: {item_list}')

I am getting the expected output for the simpler version:

apple   Count: 4
orange  Count: 3
peach   Count: 2
banana  Count: 1
plum    Count: 1

However when I try to add the item_number list to it, the results are no longer sorted which plays havoc when I use a most_common() value smaller than the total number of fruit varieties:

plum    Count: 1    List: [11]
banana  Count: 1    List: [5]
orange  Count: 3    List: [3, 7, 10]
apple   Count: 4    List: [2, 4, 6, 9]
peach   Count: 2    List: [1, 8]

I'm sure there is something I could be doing differently here, but I'm not quite sure what.

counter python-3.x dictionary python
2 Answers

Counter(mydefaultdict) isn't doing what you think it's doing. You are feeding a defaultdict of lists to Counter, whose purpose is to count occurrences, not calculate the lengths of lists. Indeed, the values of your Counter object are just lists, not integers. Counter doesn't complain because it's a subclass of dict and like dict can be initialised with another dictionary.

To order by longest list you can use heapq.nlargest with a custom function:

from heapq import nlargest

for fruit, item_list in nlargest(10, mydefaultdict.items(), key=lambda x: len(x[1])):
    print(f'{fruit}\tCount: {len(item_list)}\tList: {item_list}')

apple   Count: 4    List: [2, 4, 6, 9]
orange  Count: 3    List: [3, 7, 10]
peach   Count: 2    List: [1, 8]
banana  Count: 1    List: [5]
plum    Count: 1    List: [11]
user_92095460

This part is difficult:

Counter(mydefaultdict)

Your object mydefaultdict is already populated with lists as the values, but Counter objects generally have positive integers as the values. This is not actually an error, because Counter is a dict subclass, so it will accept any dict as an initializer argument. Except there's a problem: most_common is no longer returning sane results (in case you were curious, it's actually placing a lexicographical order based on the lists).

Perhaps clearer would be something like this:

most_common_fruits = sorted(mydefaultdict, key=lambda f: len(mydefaultdict[f]), reverse=True)
for fruit in most_common_fruits:
    item_list = mydefaultdict[fruit]
    ...

Now the output is like this:

apple   Count: 4    List: [2, 4, 6, 9]
orange  Count: 3    List: [3, 7, 10]
peach   Count: 2    List: [1, 8]
banana  Count: 1    List: [5]
plum    Count: 1    List: [11]
user_6740390
Related