Python Pandas Move Na or Null values to a new dataframe

user_5389586 0
Viewed 713

I know I can drop NaN rows from a DataFrame with df.dropna(). But what if I want to move those NaN rows to a new DataFrame? Dataframe looks like

FNAME, LNAME, ADDRESS, latitude, logitude, altitude
BOB, JONES, 555 Seaseme Street, 38.00,-91.00,0.0
JOHN, GREEN, 111 Maple Street, 34.00,-75.00,0.0
TOM, SMITH, 100 A Street, 20.00,-80.00,0.0
BETTY, CROCKER, 5 Elm Street, NaN,NaN,NaN

I know I can group and move to a new DataFrame like this

grouped = df.groupby(df.FNAME)
df1 = grouped.get_group("BOB")

and it will give me a new DataFrame with FNAME of BOB but when I try

grouped = df.groupby(df.altitude)
df1 = grouped.get_group("NaN")

I get a KeyError: 'NaN'. So how can I group by Nan or Null values?

pandas dataframe python-3.x
2 Answers

Assuming you're satisfied that all 'Nan' values in a column are to be grouped together, what you can do is use DataFrame.fillna() to convert the 'Nan' into something else, to be grouped.

df.fillna(value={'altitude':'null_altitudes'}

This fills every null in the altitude column with the string 'null_altitudes'. If you do a groupby now, all 'null_altitudes' will be together. You can fill multiple columns at once using multiple key value pairs: values = {'col_1':'val_1', 'col_2':'val_2', etc}

user_127054810

You can use isna with any on rows:

# to get rows with NA in a new df
df1 = df[df.isna(axis=1).any()]
user_92992590
Related