Python: when trying to extract certain keys, how can I avoid a KeyError when in some dict elements, the key value is missing from APi json?

Ed Vermaak 0
Viewed 33

I can successfully extract every column using Python, except the one I need most (order_id) from an API generated json that lists field reps interactions with clients.

Not all interactions result in orders; there are multiple types of interactions. I know I will need to add the flag to show 'None' and then in my for loop and an if-statement to check whether the order_id is null or not. If not 'None/null', add it to the list.

I just cannot figure it out so would appreciate every bit of help!

This is the code that works:

import requests
import json

r = requests.get(baseurl + endpoint + '?page_number=1' + '&page_size=2', headers=headers)

output = r.json()

interactions_list = []

for item in output['data']:
    columns = {
        'id': item['id'],
        'number': item['user_id'],
        'name': item['user_name'],        
    }
        
    interactions_list.append(columns)  

print(interactions_list)

This returns an error-free result:

[{'id': 1, 'number': 6, 'name': 'Johnny'}, {'id': 2, 'number': 7, 'name': 'David'}]

When I include the order_id in the loop:

interactions_list = []

for item in output['data']:
    columns = {
        'id': item['id'],
        'number': item['user_id'],
        'name': item['user_name'],
        'order': item['order_id'],
    }
        
    interactions_list.append(columns)

It returns:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_17856/1993147086.py in <module>
      6         'number': item['user_id'],
      7         'name': item['user_name'],
----> 8         'order': item['order_id'],
      9     }
     10 

KeyError: 'order_id'
keyerror dictionary api json python
2 Answers

Use the get method of the dictionary:

columns = {
    'id': item.get('id'),
    'number': item.get('user_id'),
    'name': item.get('user_name'),
    'order': item.get('order_id'),
}

This will set your missing values to None. If you want to choose what the None value is, pass a second argument to get e.g. item.get('user_name', 'N/A')

EDIT: To conditionally add items based on the presence of the order_id

interactions_list = []
for item in output['data']:
    if 'order_id' in item:
        columns = {
            'id': item.get('id'),
            'number': item.get('user_id'),
            'name': item.get('user_name', 'N/A'),
            'order': item.get('order_id'),
        }
        interactions_list.append(columns)
Crapicus0

Alternatively, you can use a list comprehension approach, which should be slightly more efficient than using list.append in a loop:

output = {'data': [{'order_id': 'n/a', 'id': '123'}]}

interactions_list = [
    {
        'id': item.get('id'),
        'number': item.get('user_id'),
        'name': item.get('user_name', 'N/A'),
        'order': item.get('order_id'),
    } for item in output['data'] if 'order_id' in item
]

# [{'id': '123', 'number': None, 'name': 'N/A', 'order': 'n/a'}]
rv.kvetch0
Related