Using the json_normalize function when a key is missing

user_3396351 0
Viewed 1036

I have come across an issue with the json_normalize function. When specifying a key which may be missing for an item, it throws a key error. As you can see, listPeople does not always exist in the file.

df = {'Links':[{'id' : 1,'Gender' : 'X'},
         {'id' : 2,'Gender' : 'Y','listPeople' : [{'Person':'John', 'Age' : 42}] }
         ]
        }

test = json_normalize(df, record_path= "listPeople", errors = "ignore")
print(test)

According to the documentation, using errors = "ignore" should do the trick, but this doesn't seem to be working?

Expected Output:

Person   Age
  NULL  NULL
  John    42
json-normalize pandas dataframe python
1 Answers

errors = "ignore" works for missing dict keys, but you are missing a list that could contain multiple dicts, and that you want to pass as record_path. You could fill the missing listPeople with empty values:

[i.update({'listPeople':[{'Person':None,'Age':None}]}) for i in df['Links'] if 'listPeople' not in i.keys()]

test = pd.json_normalize(df['Links'], record_path=['listPeople'], meta=['id','Gender'], errors = "ignore")

Result:

Person Age id Gender
0 None NaN 1 X
1 John 42 2 Y
user_113807950
Related