How to improve nested for loops over DataFrame going through all possible column combinations in python?

MauroL 0
Viewed 26

I have a DataFrame and I'm using nested for loops to go through all available combinations of certain columns. I created an exemplifying code:

from pandas import DataFrame
from numpy import unique

df = DataFrame([[30, 'DEV1', 'X4Y4', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]],
                [30, 'DEV1', 'X5Y5', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]],
                [30, 'DEV2', 'X4Y4', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]],
                [30, 'DEV2', 'X5Y5', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]],
                [85, 'DEV1', 'X4Y4', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]],
                [85, 'DEV1', 'X5Y5', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]],
                [85, 'DEV2', 'X4Y5', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]],
                [85, 'DEV2', 'X5Y5', [0, 1, 2, 3], [1E-5, 2E-5, 3E-5, 4E-5]]],
               columns=['Temperature', 'Device', 'Coordinate', 'Voltage', 'Current'])

Temperature = unique(df['Temperature'])
for temperature in Temperature:
    df1 = df.query("Temperature == @temperature")
    Device = unique(df1['Device'])
    for device in Device:
        df2 = df1.query("Device == @device")
        Coordinate = unique(df2['Coordinate'])
        for coordinate in Coordinate:
            df3 = df2.query("Coordinate == @coordinate")
            # do something with df3['Voltage'] and df3['Current']

I'm sure there is a better way to do this. Online I was reading about using groupby and agg but I didn't quite get how to apply it to my case.

Could you please share your ideas?

Thank you!

dataframe python
1 Answers

You can first apply groupy function and then use groups property of it. It will return the dict having the key values all possible combinations of columns values and values of dict are labels.

list(df.groupby(["Temperature","Device","Coordinate"]).groups.keys())


[(30, 'DEV1', 'X4Y4'),
 (30, 'DEV1', 'X5Y5'),
 (30, 'DEV2', 'X4Y4'),
 (30, 'DEV2', 'X5Y5'),
 (85, 'DEV1', 'X4Y4'),
 (85, 'DEV1', 'X5Y5'),
 (85, 'DEV2', 'X4Y5'),
 (85, 'DEV2', 'X5Y5')]
userhc0k0
Related