Export multiple csv files containing only rows containing same ID values from original dataframe

user_18177705 0
Viewed 53

I'm wanting to create separate dataframes taken from a csv file, and each row with the same ID to a new dataframe (or csv file)... the list of IDs would be unknown unless I open the csv file which contains thousands of IDs... I don't necessarily need separate dataframes for each ID but I do need separate csv files for each ID... the csv could be named after the batch and saved in the same file path as the df1 source.

df1source: ID A B 0 B345 male 12 1 B980 female 34 2 B980 female 44 3 B345 female 04 4 B456 male 78

import pandas as pd

df1 = pd.read_csv(r"C:file\path\df1source.csv")

desired output:

df2 = ID A B 0 B345 male 12 1 B345 female 04

df3 = ID A B 0 B980 female 34 1 B980 female 44

df4 = ID A B 0 B456 male 78

pandas dataframe csv python loops
1 Answers

You can mask your DataFrame similar to numpy. mask = df['<column>']==<value> or mask = df.<column>==<value> and mask your DataFrame with df[mask]

Example

import pandas as pd

l = 10
df = pd.DataFrame({'ID': np.arange(l)%4,  # %4 to get 4 IDs
                   'y': np.random.random(l)})

df_id_list = []
for i in df.ID.unique():
    df_id_list.append(df[df.ID==i])
    
df_id_list[0]
ID y
0 0 0
4 0 4
8 0 8

Alternative if IDs are unique anyhow

If your IDs are anyhow unique, there is probably no need to split the DataFrame into the IDs. Each ID is represented by a row of the DataFrame i.e. a pd.Series. To get access to a row by ID you can set the ID column as index.

l = 10
df = pd.DataFrame({'ID': np.arange(l)**2, 'y': np.arange(l)})
df.set_index("ID", inplace=True)

df.loc[9].y
# 3
user_153305390
Related