Pandas, drop duplicate Name Rows, if the name is more than 20 characters

user_14032800 0
Viewed 74

I have the following data frame called df:

Name     State
Down     NY
Down     NY
Down     NY
Next In  NJ
Next In  NJ
Next In  NJ

I want to be able to drop duplicates rows based on Name only if the name is more than 5 characters long. Is this possible with drop_duplicates?

pandas dataframe python-3.x python
3 Answers

First find rows that str.len () > 5 and drop_duplicated these row then append with other rows. Try this:

df = pd.DataFrame({
    'Name': ['Down','Down','Down','Next In','Next In','Next In'],
    'State': ['NY','NY','NY','NY','NY','NY']
})

mask = (df['Name'].str.len() > 5)
df[mask].drop_duplicates().append(df[~mask]).sort_index()

Output:

    Name    State
0   Down    NY
1   Down    NY
2   Down    NY
3   Next In NY
user_17405770

You can drop_duplicates() only on rows where Name has a length of more than 5, and pd.cocnat() it with the other part of the DataFrame:

pd.concat([df[df.Name.str.len()<=5],df[df.Name.str.len()>5].drop_duplicates()],axis=0)

      Name State
0     Down    NY
1     Down    NY
2     Down    NY
3  Next In    NJ
user_91673820

You can also drop that rows where value in column "Name" exists in more than one row and the len of Name is greater than 5 using function 'drop' from data frame. Axis = 0 means that we will drop rows, inplace= True means we will work on variable df.

df = pd.DataFrame({'Name':['Down', 'Down', 'Down', 'Next In', 'Next In', 'Next 
In'], 'State':['NY', 'NJ', 'NY', 'NY', 'NJ', 'NJ']})

#Iterate over all rows
for ind in df.index:
   #checks if name is greater than 5 and checks number of occurences of the name in whole dataframe
    if (len(df.iloc[ind]["Name"])>5 and df.value_counts(subset=['Name'])[df.iloc[ind]["Name"]]>1):
        df.drop(ind, axis=0, inplace=True)
user_118817530
Related