Count elements in pandas dataframe columns

user_7150108 0
Viewed 1471

Any suggestions for elegant ways to make the following counts for each column in my dataframe:

  • Count of elements in the column which are numeric (any value)
  • Count of elements in the column which are numeric and nonzero

Here's some code to generate a dataframe like mine (but much smaller):

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
#from matplotlib import style
import seaborn as sns

df = pd.DataFrame(data=np.random.random_integers(low=0,high=10,size=(50,1)),
                  columns=['all']
                  )

# set up the grouping
g = df.groupby(pd.cut(df['all'],[-1,2,4,6,8,10]))
for name, data in g.groups.items():
    df[name] = df.loc[data]['all']

This produces a dataframe that looks like the following:

dataframe

pandas dataframe python
2 Answers

You can do this without the apply:

In [11]: df.notnull().sum()
Out[11]:
all        50
(-1, 2]    12
(2, 4]     12
(4, 6]      7
(6, 8]     10
(8, 10]     9
dtype: int64

In [12]: (df.notnull() & (df != 0)).sum()
Out[12]:
all        46
(-1, 2]     8
(2, 4]     12
(4, 6]      7
(6, 8]     10
(8, 10]     9
dtype: int64

Note: This works as counting True values is the same as taking their sum.

user_12402680

Count of elements in the column which are numeric (any value)

df.apply(lambda x : x.notnull().sum())
Out[262]: 
all        50
(8, 10]     9
(-1, 2]    12
(4, 6]      9
(2, 4]     12
(6, 8]      8
dtype: int64

Count of elements in the column which are numeric and nonzero

df.mask(df==0).apply(lambda x : x.notnull().sum())
Out[263]: 
all        48
(8, 10]     9
(-1, 2]    10
(4, 6]      9
(2, 4]     12
(6, 8]      8
dtype: int64
user_79645270
Related