How to run code for multiple dataframe items

user_11788255 0
Viewed 60

I have the following dataframe: 

      df_Station = pd.DataFrame({'Name': ['Station 1024', 'Station 1024',
                              'Station 1024', 'Station 1024',
                              'Station 5588', 'Station 5588',
                              'Station 1211', 'Station 1211',
                              'Station 1211', 'Station 1200'], 
                       'Type': ['Weather_type1', 'Weather_type2',
                                'Weather_type2', 'Weather_type3',
                                'Weather_type1', 'Weather_type1',
                                'Space_type4', 'Space_type5',
                               'Space_type4', 'Space_type5']})


      print(df_Station)

               Name            Type
          Station 1024    Weather_type1
          Station 1024    Weather_type2
          Station 1024    Weather_type2
          Station 1024    Weather_type3
          Station 5588    Weather_type1
          Station 5588    Weather_type1
          Station 1211    Space_type4
          Station 1211    Space_type5
          Station 1211    Space_type4
          Station 1200    Space_type5

I apply one filter, according to a constant that is defined:

       cteName = 'Station 1024'
       df_Station_Specific_Name = df_Station[df_Station['Name'] == cteName]

I apply another filter, according to a constant that is defined:

       cteType = 'Weather_type2'
       df_Station_Specific_Name_Type = df_Station_Specific_Name[df_Station_Specific_Name['Type'] == 
                                       cteType]

Then, I do several operations on the filtered dataframe. For example, inserting a new column:

       df_Station_Specific_Name_Type['New_Columns'] = 1

       print(df_Station_Specific_Name_Type)

            Name              Type       New_Columns
         Station 1024   Weather_type2         1
         Station 1024   Weather_type2         1

Finally, I save this final dataframe. The file of type csc will contain the name of the two constants for easy identification.

          df_Station_Specific_Name_Type.to_csv('Result_%s_%s' %(cteName, cteType))

The code works perfectly. My question is as follows:

I would like to generate several outgoing dataframes, as changing the two constants, that is, making a combination between them.

For example, for cteName = Station 1024 I would like to run for all unique types (Weather_type1, Weather_type2 ...)

Would there be an automatic way to do this? without having to keep changing the constants cteName and cteType.

Note: the operation performed, adding a column, is just an example. After filtering the dataframe, several operations are performed. I am using Jupyter Anaconda line execution.

Thanks for the help.

pandas python
2 Answers

You could use groupby and loop over each groups:

for (cteName, cteType), df_ in df_Station.groupby(['Name', 'Type']):
    #to avoid SettingWithCopyWarning
    df_ = df_.copy()
    # do your stuff
    df_['New_Columns'] = 1
    # save to CSV
    df_.to_csv('Result_%s_%s' %(cteName, cteType))

if you have a list of couples (cteName, cteType) you want, you could add a if like:

l_couples = [('Station 1024', 'Weather_type2')]
for (cteName, cteType), df_ in df_Station.groupby(['Name', 'Type']):
    if (cteName, cteType) in l_couples:
        #to avoid SettingWithCopyWarning
        df_ = df_.copy()
        # do your stuff
        df_['New_Columns'] = 1
        # save to CSV
        df_.to_csv('Result_%s_%s' %(cteName, cteType))
user_92747320

You could try creating a new dataframe with your unique name values, and then iterating through the new dataframe:

df_Station = pd.DataFrame({'Name': ['Station 1024', 'Station 1024',
                                    'Station 1024', 'Station 1024',
                                    'Station 5588', 'Station 5588',
                                    'Station 1211', 'Station 1211',
                                    'Station 1211', 'Station 1200'],
                           'Type': ['Weather_type1', 'Weather_type2',
                                    'Weather_type2', 'Weather_type3',
                                    'Weather_type1', 'Weather_type1',
                                    'Space_type4', 'Space_type5',
                                    'Space_type4', 'Space_type5']})

for _name, _type in zip(df_Station['Name'], df_Station['Type']):
    df_Station_Specific_Name = df_Station[df_Station['Name'] == _name]
    df_Station_Specific_Name_Type = df_Station_Specific_Name[df_Station_Specific_Name['Type'] == _type]
    df_Station_Specific_Name_Type['New_Columns'] = 1
    df_Station_Specific_Name_Type.to_csv('Result_%s_%s' % (_name, _type))
user_84566080
Related