Saving dataframes with a key

user_13769249 0
Viewed 69

I'm trying to parse a csv file and print certain timeseries graphs.

About csv file: The csv file contains a lot of data from which I need to parse a certain sections of it based on the id inside a for loop. The csv file looks like that:

ID,name,date,confirmedInfections
DE2,BAYERN,2020-02-24,19
.
DE2,BAYERN,2020-02-25,19
DE1,BADEN-WÃœRTTEMBERG,2020-02-24,1
.
DE1,BADEN-WÃœRTTEMBERG,2020-02-26,7
.
DE4,BRANDENBURG,2020-02-24,2
.
DE4,BRANDENBURG,2020-07-27,45

About my code: So, after seeing an example of the csv file I'm about to tell you what is my goal. I need to save a dataframe for every different ID appears in that csv file, with all the following columns like name,date and confirmedInfections for a particular ID. There are about 12 IDs in this file, so I'm trying to save different dataframes for each ID with a for loop and then do some actions for plotting timeseries graphs.

My code:

import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.compat import pandas

def main(file):
    id_array = ["DE2", "DE1", "DE4", "DE6", "DE5", "DE8", "DE7", "DE9", "DEB", "DEA", "DED", "DEC", "DEF", "DEE", "DEG"]
    df = pd.read_csv(file, index_col='ID')
    print(df)
    for key in id_array:
        df=df.loc[key]
        print(key)
        df.plot()
        plt.show()

main('data.txt')

After running this code I'm taking the results of ID=DE2 and the graph of it. But after DE2 my code is crashing and none of the following dataframes I want to see don't appear.

Errors:

Traceback (most recent call last):
  File "C:\Users\Deray\PycharmProjects\covid-19\venv\lib\site-packages\pandas\core\indexes\base.py", line 3080, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas\_libs\index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\index.pyx", line 96, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\index.pyx", line 120, in pandas._libs.index.IndexEngine._get_loc_duplicates
KeyError: 'DE1'

Thanks for your time, any thoughts?

pandas csv python
2 Answers

The problem is that you're assigning the key-filtered dataframe to 'df' within your 'for' loop, thus overwriting the original dataframe. To fix, you need to assign the filtered dataframe to another variable. Try:

for key in id_array:
    df_temp = df.loc[key]
    print(key)
    df_temp.plot()
    plt.show()
user_10240760

You are overiding the df in the first iteration.

df=df.loc[key]

You can save the raw to different object if needed.

A better approch will be to use iterrows (iterrows docs) or itertuples (itertuples docs)

user_152139300
Related