Finding Columns with less than 3 duplications of every categorical value in a data frame python

user_9820165 0
Viewed 25

Basically, I am trying to find out the categorical columns which has less than 3 duplications of every class. For eg, In the attached screenshot, I have a column called Name, 'Type 1' and I want these column names to be returned in my output if each category in these columns (Say Bulbasaur, Nidoino) are present < 3 times and the reason why I want to perform this is in such cases where a column mostly has unique values, I can label encode them.

Below is the code I tried out, but I am not finding the solution.

Could someone please help?

labels = [str(Training_data[object_cols].unique()[i]) for i in range(Training_data[object_cols].nunique())]
values = [Training_data[object_cols].value_counts()[i] for i in range(Training_data[object_cols].nunique())]
labels
values
Target_dis = pd.DataFrame((labels, values))
Target_dis = Target_dis.T
Target_dis.columns = ['Labels', 'Counts']
Target_dis = Target_dis.sort_values('Counts', ascending=False)
Target_dis

enter image description here

dataframe duplicates for-loop python
2 Answers

You can use the value_counts method on your columns:

Target_dis['Name'].value_counts()
user_25662830

I'm not sure exactly what you want, and why you would label-encode. So -- I added some code below that outputs, for each column, how many times the most common value occurs.

import pandas as pd

df = pd.DataFrame({
    'Name': ['Bulbasaur', 'Bulbasaur', 'Nidorino', 'Nidoking', 'Clefairy', 'Clefable', 
             'Vulpix', 'Ninetales', 'Jigglypuff'],
    'Type 1': ['Grass', 'Grass', 'Grass', 'Grass', 'Fire', 'Fire', 'Fire', 'Fire', 'Fire'],
    'Type 2': ['Poison', 'Poison', 'Poison', 'Poison', '', '', 'Flying', 'Dragon', 'Flying'],
    'Total': [318, 405, 525, 625, 309, 405, 534, 634, 634],
    'HP': [45, 60, 80, 80, 39, 58, 78, 78, 78],
    'Attack': [49, 62, 82, 100, 52, 64, 84, 130, 104],
})

for col in df:
    max_count = df[col].value_counts().max()
    print(f"Values in column '{col}' are repeated at most {max_count} times.")

Values in column 'Name' are repeated at most 2 times.
Values in column 'Type 1' are repeated at most 5 times.
Values in column 'Type 2' are repeated at most 4 times.
Values in column 'Total' are repeated at most 2 times.
Values in column 'HP' are repeated at most 3 times.
Values in column 'Attack' are repeated at most 1 times.
user_21117780
Related