How to avoid error when handle a JSON without an expected field

Viewed 11

I have the following JSON structure obtained from here:

[{'country': 'Afghanistan',
  'country_code': 'AFG',
  'continent': 'Asia',
  'population': 38928341,
  'indicator': 'cases',
  'weekly_count': 0,
  'year_week': '2020-01',
  'rate_14_day': '0',
  'cumulative_count': 0,
  'source': 'Epidemic intelligence national data'},
 {'country': 'Afghanistan',
  'country_code': 'AFG',
  'continent': 'Asia',
  'population': 38928341,
  'indicator': 'cases',
  'weekly_count': 0,
  'year_week': '2020-02',
  'rate_14_day': '0',
  'cumulative_count': 0,
  'source': 'Epidemic intelligence national data'},
  ...}

So I'm passing it to a transform task where the fields are separated in a list of lists to be transformed in a dataframe at the end of the process.

My problem is when some field is not present in a record, for example, in the case below where the field weekly_count is not present and the code brokes when comes to it:

{'country': 'Afghanistan', 
'country_code': 'AFG', 
'continent': 'Asia', 
'population': 38928341, 
'indicator': 'cases', 
'year_week': '2021-52', 
'rate_14_day': '0.922207293652714', 
'cumulative_count': 158156, 
'source': 'Epidemic intelligence national data'}

My code to pass from JSON to list is this:

def transform(data: dict) -> pd.DataFrame:
    transformed = []
    for record in data:
        transformed.append({
            'country': record['country'],
            'country_code': record['country_code'],
            'continent': record['continent'],
            'population': record['population'],
            'indicator': record['indicator'],
            'weekly_count': record['weekly_count'],
            'year_week': record['year_week'],
            'cumulative_count': record['cumulative_count'],
            'rate_14_day': record['rate_14_day'],
            'source': record['source']
        })
    return pd.DataFrame(transformed)
0 Answers
Related