Pythonic way to find max of columns in df within a dict

user_3158876 0
Viewed 473

I have a dictionary which contains several, identically formatted, dataframes. I would like to find the max value of a specific column in all dataframes. I could iterate through the dictionary, but I assume there must be a more pythonic way to do it.

For example, say I have two dataframes (shortened for example, but they will have more numeric columns than just age:

b = pd.DataFrame({'name': ['Ted', 'Ben', 'Frank', 'Allen'],
     'age': [22,38,26,35]})
g = pd.DataFrame({'name': ['Emily', 'Anna', 'Amy', 'Steph'],
     'age': [18,23,44,21]})

Contained within a dict:

dict = {'Boys': b,
     'Girls': g}

Then I'd assume something like this would work, but it doesn't because the dictionary is not subscriptable, which I think I understand, I'm just not sure the correct way to do it:

Max = max(dict.values()['age'])

Is there a better way without iterating through in a loop? (if it wasn't clear, the answer should be 44 in this example)

pandas dataframe max dictionary python
5 Answers

Do the following:

from itertools import chain
result = max(chain.from_iterable(df['age'] for df in d.values()))
print(result)

Output

44

Notice that I renamed the dictionary as d because you should not use built-in names such as dict.

user_40015920

Setup

b = pd.DataFrame({'name': ['Ted', 'Ben', 'Frank', 'Allen'],
     'age': [22,38,26,35]})
g = pd.DataFrame({'name': ['Emily', 'Anna', 'Amy', 'Steph'],
     'age': [18,23,44,21]})

d = {'B': b, 'G': g}

pd.concat and max(level=0)

pd.concat(d).age.max(level=0)

B    38
G    44
Name: age, dtype: int64

or just max

pd.concat(d).age.max()

44

less Pandas

This is short, but not necessarily the best. I'm suspicious of this for bigger data.

max(map(lambda x: max(d[x].age), d))

44

Generator

Because I can

def y(d):
    for x in d:
        yield from d[x].age

max(y(d))

44
user_23366540

First version

max(b.append(g)['age'])
Out[1]:
    44

Second version

dict1 = {'Boys': b, 'Girls': g}

max([max(dict1[k]['age']) for k in dict1.keys()])
Out[2]:
    44

max([max(v['age']) for v in dict1.values()])
Out[3]:
    44
user_117119530 
df = pd.concat(d.values())
m = df.max()
print(m)

name    Ted
age      44
dtype: object
user_107409660

np.nanmax Without an axis it returns the scalar maximum.

np.nanmax([df.age.to_numpy() for df in [b,g]])  #much faster on np.array than pd.Series
#44
user_43333590
Related