Get the top N values from each row of a Pandas dataframe with their respective Column names

user_13982723 0
Viewed 982
Index          Class 1               Class 2         Class 3         Class 4          Class 5  
0              0.95693475            0.252198994      0.0            0.335894585      0.611441553
1              0.473615974           0.0              0.510585248    0.5007305        0.975620011
2              0.224682823           0.122315248      0.6407305        0.0            0.872211390

This is a sample of the dataframe I am working on. I have around 200 Class's in my original dataframe and around 85000 rows and for each row of my dataframe I want to find out the top 3 Categories with their values sorted in descending order:

    Expected output:
    Row 0: [{Class 1 : 95693475}, {Class 5: 0.611441553}, {Class 4: 0.335894585}]
    Row 1: [{Class 5 : 0.975620011}, {Class 3: 0.510585248}, {Class 4: 0.5007305}]
etc etc...

Note: List and dict in the expected output are just added for reference, just need to output the top 3 scores with their category names for each row in the dataframe. Can anyone help me with this

pandas series dataframe sorting python
1 Answers

References :

Apply funtion to row in pandas - Example 3

Return first n keyvalue pairs from dict

To return the first n keyvalue pairs from dict

from itertools import islice
def take(n, iterable):
    return list(islice(iterable, n))

drop the Index Column

df.drop('Index', axis=1,inplace=True)

Function that can be applied to all rows to find the topmost 3 categories

The topN function takes as input parameter row : which would be a row of the dataframe and n : denoting how many topmost elements you want to extract.

def topN(row, n):
    x = row.to_dict() # convert the input row to a dictionary 
    x = {k: v for k, v in sorted(x.items(), key=lambda item: -item[1])} # sort the dictionary based on their values 
    n_items = take(n, x.items()) # extract the first n values from the dictionary 
    return n_items
n = 3 #number of elements needed
df['X'] = df.apply(lambda row : topN(row,n), axis = 1)

Output :

A new column X with the required result as dictionaries is stored. You can convert the column into an array too.

Class 1 Class 2 Class 3 Class 4 Class 5 X
0   0.956935    0.252199    0.000000    0.335895    0.611442    [(Class 1, 0.95693475), (Class 5 , 0.61144155...
1   0.473616    0.000000    0.510585    0.500731    0.975620    [(Class 5 , 0.975620011), (Class 3, 0.5105852...
2   0.224683    0.122315    0.640730    0.000000    0.872211    [(Class 5 , 0.87221139), (Class 3, 0.6407305)...

Example to remove all values with 0.0 :

d = {1:0.0, 2:0.0, 3:1.0}
x={k:v for k,v in d.items() if v}
x # prints {3: 1.0}
user_100773540
Related