I have defined a Pandas dataframe which contains a column name as 'emp_title'. I want to see the total number of unique entries in that specific column.
I used:
len(df['emp_title'].unique())
which gives me a value of 173106
whereas when I use:
df['emp_title'].nunique()
it gives me a value of 173105 which should be the actual size.
Can any one explain why I shouldn't be using the code with the len() function. Or is there probably an issue with the dataset at play here?
It means there is some missing value(s), because Series.nunique by default remove missing values.
So for same output change default parameter dropna=True to False:
df['emp_title'].nunique(dropna=False)
The difference is that len the length of all elements in a list / array, and unique() returns of all unique elements including missing ones nan, whereas nunique() doesn't take into accoung nan:
df = pd.DataFrame({'A':[1,2,3,4,np.nan]})
print(df['A'].nunique()
Rightfully returns 4, and
print(df['A'].unique())
Returns the list of all unique elements:
[ 1. 2. 3. 4. nan]
Which has a length of 5.