Pandas - Column Addition for duplicate or list values in a column in Dataframe

user_10919961 0
Viewed 85

Lets say a dataframe looks like this

  one  two 
a  1.0  1.0 
b  2.0  2.0 
c  3.0  3.0 
d  NaN  4.0

Adding new three column is like this

df["three"] = df["one"] * df["two"]

Result

   one  two     three 
 a  1.0  1.0    1.0 
 b  2.0  2.0    4.0 
 c  3.0  3.0    9.0   
 d  NaN  4.0    NaN

How about colum values that contains duplicate lists or list, and I need to create a new column and add the number with highest value

Example

    one  two 
 a  1.0  [12,1]
         [12,1]
 b  2.0  2.0    
 c  3.0  3.0    
 d  NaN  4.0

So I want like this

    one  two        flag
 a  1.0  [12,1]      12
         [12,1]
 b  2.0  [200,400]   400
 c  3.0  3.0         3.0
 d  NaN  4.0         4.0

Thanks

pandas python
1 Answers

If there is list or nested lists or floats you can flatten lists with max:

df = pd.DataFrame({"two":  [[[12,1],[12,1]] ,[200,400] ,3.0,4.0 ]})
    
from typing import Iterable 
              
#https://stackoverflow.com/a/40857703/2901002
def flatten(items):
    """Yield items from any nested iterable; see Reference."""
    for x in items:
        if isinstance(x, Iterable) and not isinstance(x, (str, bytes)):
            for sub_x in flatten(x):
                yield sub_x
        else:
            yield x
            
df['new'] = [max(flatten(x)) if isinstance(x, list) else x for x in df['two']]
print (df)
                  two    new
0  [[12, 1], [12, 1]]   12.0
1          [200, 400]  400.0
2                 3.0    3.0
3                 4.0    4.0

EDIT: For max in new DataFrame for all columns use aggregate function max:

df = df_orig.pivot_table(index=['keyword_name','volume'], 
                    columns='asin', 
                    values='rank', 
                    aggfunc=list)

df1 = df_orig.pivot_table(index=['keyword_name','volume'], 
                     columns='asin', 
                     values='rank', 
                     aggfunc='max')

out = pd.concat([df, df1.add_suffix('_max')], axis=1)
user_29010020
Related