so here's my problem:
a have the following architecture:
Repository:
-Sample A folder:
-file_A_quant.csv
-file_A_isdb.tsv
-WORKSPACE_A folder:
-file_A_sir.tsv
-Sample B folder:
-file_B_quant.csv
-file_B_isdb.tsv
-WORKSPACE_B folder:
-file_B_sir.tsv
-Sample N folder:
-file_N_quant.csv
-file_N_isdb.tsv
-WORKSPACE_N:
-file_N_sir.tsv
I am able to apply a particular function to certain of the files in each folder, they're recognized by a particular suffix. i do this using this kind of function:
def ind_quant_table(repository_path, quant_suffix):
for r, d, f in os.walk(repository_path):
for file in (f for f in f if f.endswith(quant_suffix)):
complete_file_path =r+'/'+file
df = pd.read_csv(complete_file_path)
df.rename(columns = lambda x: x.replace(' Peak area', ''),inplace=True)
df.rename(columns = lambda x: x.replace(file_extention, ''),inplace=True)
df.drop(list(df.filter(regex = 'Unnamed:')), axis = 1, inplace = True)
df.sort_index(axis=1, inplace=True)
prefix = 'treated_'
df.to_csv(r+'/'+prefix+file, sep =',')
basically, with the function, I process the data in the way I want and then save it as a new file.
Now, in the same way i treat the other files, and i get individual files.
HOW can i, for each sample folder, treat the individual files and merge them and then save one unique file?. The files shared a common column that can be used to merge the information.
so far i tried something like this, but it does not work:
def ind_quant_table_f(repository_path, quant_suffix, isdb_suffix, sir_suffix):
for r, d, f in os.walk(repository_path):
for file in (f for f in f if f.endswith(quant_table_suffix)):
complete_file_path =r+'/'+file
df = pd.read_csv(complete_file_path)
'do something to the df'
df
for r, d, f in os.walk(repository_path):
for file in (f for f in f if f.endswith(isdb_suffix)):
complete_file_path =r+'/'+file
dfisdb = pd.read_csv(complete_file_path)
'do something to the dfisdb'
df = pd.merge (df, dfisdb)
for r, d, f in os.walk(repository_path):
for file in (f for f in f if f.endswith(sir_suffix)):
complete_file_path =r+'/'+file
dfsii = pd.read_csv(complete_file_path)
'do something to the dfsir'
df = pd.merge (df, dfsir)
prefix = 'treated_'
df.to_csv(r+'/'+prefix+file, sep =',')
i would appreciate your ideas!
Luis