Filter nan values out of rows in pandas

user_13750269 0
Viewed 1964

I am working on a calculator to determine what to feed your fish as a fun project to learn python, pandas, and numpy.

My data is organized like this:

enter image description here

As you can see, my fishes are rows, and the different foods are columns.

What I am hoping to do, is have the user (me) input a food, and have the program output to me all those values which are not nan.

The reason why I would prefer to leave them as nan rather than 0, is that I use different numbers in different spots to indicate preference. 1 is natural diet, 2 is ok but not ideal, 3 is live only.

Is there anyway to do this using pandas? Everywhere I look online helps me filter rows out of columns, but it is quite difficult to find info on filter columns out of rows.

Currently, my code looks like this:

import pandas as pd
import numpy as np


df = pd.read_excel(r'C:\Users\Daniel\OneDrive\Documents\AquariumAiMVP.xlsx')
clownfish = df[0:1]
angelfish = df[1:2]
damselfish = df[2:3]

So, as you can see, I haven't really gotten anywhere yet. I tried filtering out the nulls using, the following idea:

clownfish_wild_diet = pd.isnull(df.clownfish)

But it results in an error, saying:

AttributeError: 'DataFrame' object has no attribute 'clownfish'

Thanks for the help guys. I'm a total pandas noob so it is much appreciated.

pandas data-manipulation dataframe python
1 Answers

You can use masks in pandas:

food = 'Amphipods'
mask = df[food].notnull()
result_set = df[mask]

df[food].notnull() returns a mask (a Series of boolean values indicating if the condition is met for each row), and you can use that mask to filter the real DF using df[mask].

Usually you can combine these two rows to have a more pythonic code, but that's up to you:

result_set = df[df[food].notnull()]

This returns a new DF with the subset of rows that meet the condition (including all columns from the original DF), so you can use other operations on this new DF (e.g selecting a subset of columns, drop other missing values, etc)

See more about .notnull(): https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.notnull.html

user_62692680
Related