Dropping rows, where a dynamic number of integer columns only contain 0's

user_9464975 0
Viewed 36

I have the following problem - I have the following example data-frame:

Name Planet Number Column #1 Number Column #2
John Earth 2 0
Peter Terra 0 0
Anna Mars 5 4
Robert Knowhere 0 1

Here, I want to only remove the rows, in which there are numbers, that are all 0's. In this case this is the second row. So my data-frame has to become like this:

Name Planet Number Column #1 Number Column #2
John Earth 2 0
Anna Mars 5 4
Robert Knowhere 0 1

For this, I have a solution and it is the following:

 new_df = old_df.loc[(a['Number Column #1'] > 0) + (a['Number Column #2'] > 0)]

This works, however I have another problem. My dataframe, based on the request, will dynamically have a different number of number columns. For example:

Name Planet Number Column #1 Number Column #2 Number Column #3
John Earth 2 0 1
Peter Terra 0 0 0
Anna Mars 5 4 2
Robert Knowhere 0 1 1

This is the problematic part, as I am not sure how I can adjust my code to work for dynamic columns. I've tried multiple things from StackOverflow and the Pandas documentation - however most examples only work for dataframes, in which all columns are integers. Pandas would them consider them booleans, and you can add a simple solution like this:

 new_df = (df != 0).any(axis=1)

In my case however, the text columns, which are always the same, are the problematic ones. Does anyone have an idea for a solution here? Thanks a lot in advance!

P.S. I have the names of the number columns available beforehand in the code as a list, for example:

 my_num_columns = ["Number Column #1", "Number Column #2", "Number Column #3"]
 # my pandas logic...
pandas dataframe python
1 Answers

IIUC:

You can try via select_dtypes() and select int and float columns after that check your condition and filter out your dataframe:

df=df.loc[~df.select_dtypes(['int','float']).eq(0).all(axis=1)]
#OR
df=df.loc[df.select_dtypes(['int','float']).ne(0).any(axis=1)]

Note: If needed you can also include 'bool' columns and typecast it to float and then check your condition:

df=df.loc[df.select_dtypes(['int','float','bool']).astype(float).ne(0).any(axis=1)]
user_142898920
Related