I have several csv files that I would like to combine and format using python. I was able to do what I need using 2 files, however when I try to loop trhoug all hundreds of files I need to combine, I can't quite figure out the loop.
Here is what I am need to do for all files in the folder
MOZ03039E11=pd.read_csv("MOZ03039E11_protozoa.csv", header=None)
MOZ03039E11.columns=["Parasites","MOZ03039E11"]
MOZ03039E11=MOZ03039E11.set_index('Parasites')
MOZ03027F04=pd.read_csv("MOZ03027F04_protozoa.csv", header=None)
MOZ03027F04.columns=["Parasites","MOZ03027F04"]
MOZ03027F04=MOZ03039E11.set_index('Parasites')
outer_merged = pd.merge(
MOZ03027F04, MOZ03039E11, how="outer", on =["Parasites"])
This gives me the result I need, which is to make the second column to each file into a column, combining the first column as index. This is how I tried to loop:
import os
import glob
import pandas as pd
fmask = '/Users/path/to/files/species/*csv'
dfs = []
for f in glob.glob(fmask):
df = pd.read_csv(f, sep=',')
df.columns=["Parasites",f]
df2=df.set_index('Parasites')
df3=pd.merge(
dfs, df2, how="outer", on =["Parasites"])
But it keeps giving me the "EmptyDataError: No columns to parse from file". All files are simple 2-column files, with names on column 1 and numbers on column 2. What am I doing wrong here?