Python : How to remove rows which has nan values for dict inside list

Assuming nan is math.nan, you can do this:

for sub_a in a:
    for key, val in sub_a.copy().items():
        try:
            if val is None or math.isnan(val):
                del sub_a[key]
        except TypeError:
            continue

The above will mutate the original a. If you want a separate dict b as the output, you can do b = a.copy() and replace a with b in the outer for.

The reason I wrapped math.isnan in a try / except is because not all values in your dicts are real numbers, some are strings. If the TypeError is thrown, we know that the value couldn't possibly be nan.

The resulting dict is:

[{'name': 'toom D',
  'status_ref': True,
  'date_status_ref': Timestamp('2018-12-17 09:29:58.864000')},
 {'name': 'russ arnold',
  'status_ref': True,
  'date_status_ref': Timestamp('2018-12-17 09:29:58.864000')},
 {'name': 'cart wright',
  'status_ref': True,
  'date_status_ref': Timestamp('2018-12-17 09:29:58.864000'),
  'expiry': 'no',
  'renew': 'yes'}]

You can iterate through key/values of your dicts and filter them. like this:

a = [{'name': 'toom D',
    'status_ref': True,
    'date_status_ref': Timestamp('2018-12-17 09:29:58.864000'),
    'expiry': None,
    'renew': nan},
    {'name': 'russ arnold',
    'status_ref': True,
    'date_status_ref': Timestamp('2018-12-17 09:29:58.864000'),
    'expiry': None,
    'renew': nan},
    {'name': 'cart wright',
    'status_ref': True,
    'date_status_ref': Timestamp('2018-12-17 09:29:58.864000'),
    'expiry': 'no',
    'renew': 'yes'}]

def filter_dict(d):
    new_dict = dict()
    for (key, value) in d.items():
        if not (value is None or np.isnan(value)):
            new_dict[key] = value
    return new_dict

b = [filter_dict(d) for d in a]

Not the best solution but you can try.

a = [{'name': 'toom D',
  'status_ref': True,
  'date_status_ref': 'a',
  'expiry': None,
  'renew': nan},
 {'name': 'russ arnold',
  'status_ref': True,
  'date_status_ref': 'b',
  'expiry': None,
  'renew': nan},
 {'name': 'cart wright',
  'status_ref': True,
  'date_status_ref': 'c',
  'expiry': 'no',
  'renew': 'yes'}]

for d in a:
    for k,v in d.copy().items():
        if (str(v) in ['None', 'NaN', 'nan']):
            d.pop(k)

a:

[{'name': 'toom D', 'status_ref': True, 'date_status_ref': 'a'},
 {'name': 'russ arnold', 'status_ref': True, 'date_status_ref': 'b'},
 {'name': 'cart wright',
  'status_ref': True,
  'date_status_ref': 'c',
  'expiry': 'no',
  'renew': 'yes'}]