How to return dataframe with multiple conditions in pandas

user_12862963 0
Viewed 98

Here is the sample dataframe.

data = [['United Kingdom', 'High income'], ['Albania', 'Upper middle income'], ['Russia', 'Upper middle income'], ['Afganistan','Low income'], ['USA','High income']]
df = pd.DataFrame(data, columns = ['Country', 'Income Group'])

Here I was trying to return only countries with high income and upper middle income:

df = np.where(df['Income Group'] == 'High income' & df['Income Group'] == 'Upper middle income')

Here is the output:

TypeError: tuple indices must be integers or slices, not str

But if you use the same with other column works fine:

df = np.where(df['Country'] == 'USA')

What is the problem with column 'Income Group'?

Very appreciate for any help

pandas python
6 Answers

isin can be also useful here:

df[df['Income Group'].isin(['High income', 'Upper middle income'])]

Output:

          Country         Income Group
0  United Kingdom          High income
1         Albania  Upper middle income
2          Russia  Upper middle income
4             USA          High income
user_67927430

it seems from the sample you posted that you are missing () and you are confusing or with and so | instread of &:

data = [['United Kingdom', 'High income'], ['Albania', 'Upper middle income'], ['Russia', 'Upper middle income'], ['Afganistan','Low income'], ['USA','High income']]
df = pd.DataFrame(data, columns = ['Country', 'Income Group']) 
df = np.where((df['Income Group'] == 'High income') |(df['Income Group'] == 'Upper middle income'))
df
>>>(array([0, 1, 2, 4], dtype=int64),)

which return the indexes instead of the dataframe, if you plan to return the dataframe so do so:

df = df[(df['Income Group'] == 'High income') |(df['Income Group'] == 'Upper middle income')]
df
>>> Country Income Group
0   United Kingdom  High income
1   Albania Upper middle income
2   Russia  Upper middle income
4   USA High income
user_88938270

Another technique other than np.where, to slice dataframe based on multiple condition:

df[ (condition_1) & (condition_2) | (condition_3) ]

For your question, if you want to return countries with high income or upper middle income, you should use or (|) condition not and (&)

df_high = df[(df['Income Group'] == 'High income') | (df['Income Group'] == 'Upper middle income')]

df_high

    Country Income   Group
0   United Kingdom   High income
1   Albania          Upper middle income
2   Russia           Upper middle income
4   USA              High income
user_92983150

You need OR (|) instead of AND (&) and you need the brackets around the individual conditions

data = [['United Kingdom', 'High income'], ['Albania', 'Upper middle income'], ['Russia', 'Upper middle income'], ['Afganistan','Low income'], ['USA','High income']]
df = pd.DataFrame(data, columns = ['Country', 'Income Group']) 
df = np.where((df['Income Group'] == 'High income') | (df['Income Group'] == 'Upper middle income')) # changed to OR
df
user_11789710

This is just a missing parentheses. Because in python precedence & is stronger than ==.

You need something like:

df = np.where((df['Income Group'] == 'High income') & (df['Income Group'] == 'Upper middle income'))

update: and yes as other stated too, you need or (|) operator.

user_37006260 
data = [['United Kingdom', 'High income'], ['Albania', 'Upper middle income'], 
       ['Russia', 'Upper middle income'], ['Afganistan','Low income'], ['USA','High 
         income']]
df1 = pd.DataFrame(data, columns = ['Country', 'Income Group']) 
df2 = df[df['Income Group']=='High income']
df3 = df[df['Income Group']=='Upper middle income']
df4 = df2.merge(df3,how='outer')
print(df4)
user_122194170
Related