Use empty list as fill value for Series.fillna

user_11777356 0
Viewed 1891

I need to change the NaN values the column 'FocusColumn' to empty lists.

I tried to do it like this:

df['FocusColumn'].fillna([],inplace=True)

But it throws me the following error:

337         if validate_scalar_dict_value and isinstance(value, (list, tuple)):
338             raise TypeError( 
339                 '"value" parameter must be a scalar or dict, but 
340                 f'you passed a "{type(value).__name__}"'
341             )
TypeError: "value" parameter must be a scalar or dict, but you passed a "list"

What is the correct way to do it?

pandas series dataframe python
2 Answers

Looks like you can work around this by supplying a dict to Series.fillna.

>>> df
   FocusColumn
0          NaN
1          1.0
2          NaN
>>> df['FocusColumn'].fillna({i: [] for i in df.index})
0    []
1     1
2    []
Name: FocusColumn, dtype: object

Notes:

  1. Surprisingly, this only works for Series.fillna.
  2. For large Series with few missing values, this might create an unreasonable amount of throwaway empty lists.
  3. Tested with pandas 1.0.5.
user_36200030

You could try the following:

import pandas as pd
import numpy as np 

df = pd.DataFrame({'FocusColumn': [1, 2, 3, 4, np.NaN, np.NaN, np.NaN, np.NaN], 'Column1': [1, 2, 3, 4, 5, 6, 7, 8], 'Column2': [7, 8, 6, 5, 7, 8, 5, 4]})

df['FocusColumn'] = df['FocusColumn'].apply(lambda x: [] if np.isnan(x) else x)
print(df)

Output:

  FocusColumn  Column1  Column2
0           1        1       7
1           2        2       8
2           3        3       6
3           4        4       5
4          []        5       7
5          []        6       8
6          []        7       5
7          []        8       4
user_115960810
Related