Problems with .append and pd.concat()

user_14613494 0
Viewed 174

I'm a SUPER beginner and this is my first question. I have this code in which I have tried to add several df using .append and pd.concat(), using these options are part of the exercise, but it has been giving me lots of troubles, the last one the next error:

files_in_list= os.listdir("/content/gdrive/My Drive/simulation_data")

print(files_in_list)``` 

    def load_all_csv(files_names):
    # Comments here ...
    all_scenarios = []
    for files in files_in_list:
      df = pd.read_csv(files, index_col='Month')
      all_scenarios.append(df, ignore_index=True)
      pd.concat(all_scenarios, axis=1)
    return
    all_scenarios

    all_data = load_all_csv(files_in_list)
```print(all_data)```
____________________________________________________________________________
 TypeError                                 Traceback (most recent call last)
<ipython-input-43-10abf3efc373> in <module>()
      1 # Comments here
      2 # Comments here
----> 3 all_data = load_all_csv(files_in_list)
      4 print(all_data)

<ipython-input-42-0e449e23623a> in load_all_csv(files_names)
      4     for files in files_in_list:
      5       df = pd.read_csv(files, index_col='Month')
----> 6       all_scenarios.append(df, ignore_index=True)
      7       pd.concat(all_scenarios, axis=1)
      8     return

TypeError: append() takes no keyword arguments
___________________________________________________________

I've also tried with the loop outside of the function but it returned something like the example instead of a df with the same index 'Month'and a column for each of the files/scenarios.

           Scenario - Aircon Schedules
Month                                 
January                           5.61
February                          6.50
March                             9.70
April                            11.95
May                              16.52
June                             18.89
July                             22.13
August                           22.14
September                        20.38
October                          15.87
November                         11.71
December                          7.16,            Scenario - Cool roof
Month                          
January                    4.46
February                   5.39
March                      8.96
April                     11.73
May                       17.28
June                      20.54
July                      24.76
August                    24.97... ... ...

I need the function to give me the data in a data frame that has a 12-month index and the rest of the info in separate columns for each file/scenario.  
 
Any help will be most welcome!
pandas concat append function python
2 Answers

welcome to SO. In your code you mix up two things. Your df is a pandas.DataFrame and your all_scenarios is a python built-in list. Even though both implement the append function, the list does not take additional arguments as stated in the ERROR message. The code below is slightly corrected, as in the loop, only the dataframes are created and appended to the list, and afterwards concatenated.

def load_all_csv(file_names):
    all_scenarios = []
    for file_name in file_names:
      df = pd.read_csv(file_name)
      all_scenarios.append(df)
    all_scenarios = pd.concat(all_scenarios, axis=1)
    return all_scenarios

files_in_list= os.listdir("/content/gdrive/My Drive/simulation_data")
all_data = load_all_csv(files_in_list)
user_70307580

I believe the error comes because of a confusion between Python's native list append and pandas.DataFrame.append. I will comment the code a bit:

def load_all_csv(files_names):
   all_scenarios = []
   for files in files_names: #The name should match the parameter in the function.
        df = pd.read_csv(files, index_col='Month') #Read each file
        all_scenarios.append(df) #Create a list with all the dataframes previously read
   concat_dfs = pd.concat(all_scenarios, axis=1) #Concatenate all dfs
   return concat_dfs #Returns the concatenation of all dfs as a single dataframe
all_data = load_all_csv(files_in_list)
user_118970070
Related