Iterating over multiple dataframes

user_18545564 0
Viewed 79
##1    
M_members = [1000 , 1450, 1900]  
M = pd.DataFrame(M_members)

##2
a_h_members = [0.4 , 0.6 , 0.8 ]
a_h = pd.DataFrame(a_h_members) 

##3
d_h_members = [0.1 , 0.2 ]
d_h = pd.DataFrame(d_h_members)

As the output I want is in dataframe form:

1000 0.4 0.1
1000 0.4 0.2
1000 0.6 0.1
1000 0.6 0.2
1000 0.8 0.1
1000 0.8 0.2
1450 0.4 0.1
1450 0.4 0.2
1450 0.6 0.1
1450 0.6 0.2
1450 0.8 0.1
1450 0.8 0.2
1900 0.4 0.1
1900 0.4 0.2
1900 0.6 0.1
1900 0.6 0.2
1900 0.8 0.1
1900 0.8 0.2

I want to do this loop for more dataframes actually.

pandas dataframe python loops
3 Answers

Use itertools.product

>>> import itertools
>>> pd.DataFrame(itertools.product(*[M_members, a_h_members, d_h_members]))

       0    1    2
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2
user_25356110

Depending on your data size, expand_grid from pyjanitor may help with performance:

# pip install pyjanitor
import janitor as jn
import pandas as pd

others = {'a':M, 'b':a_h, 'c':d_h}
jn.expand_grid(others = others)

       a    b    c
       0    0    0
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2

You can drop a column level, or flatten it:

jn.expand_grid(others = others).droplevel(axis = 1, level = 1)

       a    b    c
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2
user_71757130

If you're starting from the DataFrames, you can use a repeated cross merge:

dfs = [M, a_h, d_h]

from functools import reduce

out = (reduce(lambda a,b: a.merge(b, how='cross'), dfs)
      .set_axis(range(len(dfs)), axis=1)
      )

Output:

       0    1    2
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2
user_163434640
Related