Summing values in a list of nested dictionaries given certain conditions

Viewed 88

I have a list of nested dictionaries below which includes car sales for a given car make/model/year.

    cars = 
    [{'id': 2, 'car': 
                    {'car_make':'Acura', 'car_model':'TL', 'car_year':2005},'total_sales':589}, 
    {'id': 30, 'car': 
                    {'car_make':'Acura', 'car_model':'TL', 'car_year':2004}, 'total_sales':167},
    {'id': 31, 'car': 
                    {'car_make':'Acura', 'car_model':'Integra', 'car_year':2008}, 'total_sales':200},
    {'id': 71, 'car':
                    {'car_make':'BMW', 'car_model':'5 Series', 'car_year':2011},'total_sales':824},
    {'id': 72, 'car':
                    {'car_make':'BMW', 'car_model':'5 Series', 'car_year':2001}, 'total_sales':6}]

I would like to sum total sales across all years and return a total_sales dictionary with car make, model and total sales.

    total_sales = {{'car_make': 'Acura', 'car_model': 'TL', 'total_sales': 756},
                  {'car_make': 'Acura', 'car_model': 'Integra', 'total_sales': 200},
                  {'car_make': 'BMW', 'car_model': '5 Series', 'total_sales': 830}}

Below is my code where I iterate over nested dictionaries and add sum the sales

total_sales = {'car_make':{}}

for car in cars:
    if car['car']['car_make'] in total_sales['car_make'] and car['car']['car_model'] in 
    total_sales['car_model']:
        total_sales['total_sales'] = total_sales['total_sales'] + car['total_sales']
else:
    total_sales['car_make'] = car['car']['car_make']
    total_sales['car_model'] = car['car']['car_model']
    total_sales['total_sales'] = car['total_sales']
    print(total_sales)

However, I am getting a running total instead of final total sales per car make and car model.

    {'car_make': 'Acura', 'car_model': 'TL', 'total_sales': 589}
    {'car_make': 'Acura', 'car_model': 'TL', 'total_sales': 756}
    {'car_make': 'Acura', 'car_model': 'Integra', 'total_sales': 200}
    {'car_make': 'BMW', 'car_model': '5 Series', 'total_sales': 824}
    {'car_make': 'BMW', 'car_model': '5 Series', 'total_sales': 830}

I am new to python and appreciate if anyone can explain what I am doing wrong. I also looked at other topics including iterating over nested dictionaries but nothing seem to fit my case.

4 Answers

You can use collections.defaultdict:

cars = [{'id': 2, 'car': {'car_make': 'Acura', 'car_model': 'TL', 'car_year': 2005}, 'total_sales': 589}, {'id': 30, 'car': {'car_make': 'Acura', 'car_model': 'TL', 'car_year': 2004}, 'total_sales': 167}, {'id': 31, 'car': {'car_make': 'Acura', 'car_model': 'Integra', 'car_year': 2008}, 'total_sales': 200}, {'id': 71, 'car': {'car_make': 'BMW', 'car_model': '5 Series', 'car_year': 2011}, 'total_sales': 824}, {'id': 72, 'car': {'car_make': 'BMW', 'car_model': '5 Series', 'car_year': 2001}, 'total_sales': 6}]
from collections import defaultdict
d = defaultdict(int)
for i in cars:
   d[(i['car']['car_make'], i['car']['car_model'])] += i['total_sales']

r = [{'car_make':a, 'car_model':b, 'total_sales':c} for (a, b), c in d.items()]

Output:

[{'car_make': 'Acura', 'car_model': 'TL', 'total_sales': 756}, 
 {'car_make': 'Acura', 'car_model': 'Integra', 'total_sales': 200}, 
 {'car_make': 'BMW', 'car_model': '5 Series', 'total_sales': 830}]

Depending how crazy your dictionaries are getting and how granular you want to get on this type of statistical info, you may want to look into refactoring your structure to use pandas which opens a lot more doors for data manipulation, slicing, etc. However for a beginner it does have a bit of a learning curve so I'll avoid the usual suggestion "hey just use this library". You can read some of the docs if you're interested, https://pandas.pydata.org/docs/

To directly answer your question, you may want to restructure your total sales dictionary. You could use a "total_sales_by_make_and_model" dictionary, and then use the make keys listed in your cars list as the keys for this dictionary -- it makes it a bit more legible, however I'm not sure how this jives with your overall "business" logic.

I would also suggest expanding out your code a bit to store key names in variables, at least until you can see where your summation is going wrong.

cars = [
    {'id': 2, 'car': {'car_make':'Acura', 'car_model':'TL', 'car_year':2005},'total_sales':589},
    {'id': 30, 'car': {'car_make':'Acura', 'car_model':'TL', 'car_year':2004}, 'total_sales':167},
    {'id': 31, 'car': {'car_make':'Acura', 'car_model':'Integra', 'car_year':2008}, 'total_sales':200},
    {'id': 71, 'car': {'car_make':'BMW', 'car_model':'5 Series', 'car_year':2011},'total_sales':824},
    {'id': 72, 'car': {'car_make':'BMW', 'car_model':'5 Series', 'car_year':2001}, 'total_sales':6}
]

total_sales_by_make_and_model = {}

for car in cars:
    car_make = car['car']['car_make']
    car_model = car['car']['car_model']
    total = car['total_sales']

    # if the make key doesn't exist, create it
    if car_make not in total_sales_by_make_and_model:
        total_sales_by_make_and_model[car_make] = {}

    # use get to set the value to "total" if the key doesn't exist
    # otherwise this will increment the current value by "total"
    total_sales_by_make_and_model[car_make][car_model] = total_sales_by_make_and_model[car_make].get(car_model, 0) + total

This result will give you the flexibility to query by model within a make key. Example:

print(total_sales_by_make_and_model['Acura'])
print(total_sales_by_make_and_model['BMW']['5 Series'])

Output:

{'TL': 756, 'Integra': 200}
830

If you then wanted to get the total sales for each make, you could use a simple loop to go through each key and sum the values:

for make in total_sales_by_make_and_model:
    total = sum(total_sales_by_make_and_model[make].values())
    print(f"Total sales for '{make}': {total}")

Output:

Total sales for 'Acura': 956
Total sales for 'BMW': 830

Edit: had to fix a few major typos that were giving the wrong sums :)

Here is one way to do it:

models=set([(i['car']['car_make'], i['car']['car_model']) for i in cars])

d={i:0 for i in models}

for i in cars:
    d[(i['car']['car_make'], i['car']['car_model'])]+=i['total_sales']
    
total_sales=[{'car_make':key[0], 'car_model':key[1], 'total_sales':val} for key, val in d.items()]

print(total_sales)

Output

[{'car_make': 'BMW', 'car_model': '5 Series', 'total_sales': 830}, {'car_make': 'Acura', 'car_model': 'TL', 'total_sales': 756}, {'car_make': 'Acura', 'car_model': 'Integra', 'total_sales': 200}]
import operator
import pprint
from itertools import groupby

cars =  [{'id': 2, 'car': 
                    {'car_make':'Acura', 'car_model':'TL', 'car_year':2005},'total_sales':589}, 
    {'id': 30, 'car': 
                    {'car_make':'Acura', 'car_model':'TL', 'car_year':2004}, 'total_sales':167},
    {'id': 31, 'car': 
                    {'car_make':'Acura', 'car_model':'Integra', 'car_year':2008}, 'total_sales':200},
    {'id': 71, 'car':
                    {'car_make':'BMW', 'car_model':'5 Series', 'car_year':2011},'total_sales':824},
    {'id': 72, 'car':
                    {'car_make':'BMW', 'car_model':'5 Series', 'car_year':2001}, 'total_sales':6}]

def map_car(item):
    item['car']['total_sales'] = item['total_sales']
    return item['car']

def get_total(key, item):
    car_make, car_model = key
    total = sum(i['total_sales'] for i in item)

    return {'car_make': car_make, 'car_model':car_model,  'total_sales': total}


mapped_car = list(map(map_car, cars))
key=operator.itemgetter('car_make', 'car_model')
grouped_cars = groupby(sorted(mapped_car, key=key), key=key)
total = [get_total(key, item) for key, item in grouped_cars]
pprint.pprint(total)

result:

[{'car_make': 'Acura', 'car_model': 'Integra', 'total_sales': 200},
 {'car_make': 'Acura', 'car_model': 'TL', 'total_sales': 756},
 {'car_make': 'BMW', 'car_model': '5 Series', 'total_sales': 830}]
Related