Reading through and editing csvs using for loop in python

Viewed 26

I have several csv files that I would like to combine and format using python. I was able to do what I need using 2 files, however when I try to loop trhoug all hundreds of files I need to combine, I can't quite figure out the loop.

Here is what I am need to do for all files in the folder

MOZ03039E11=pd.read_csv("MOZ03039E11_protozoa.csv", header=None)
MOZ03039E11.columns=["Parasites","MOZ03039E11"]
MOZ03039E11=MOZ03039E11.set_index('Parasites')

MOZ03027F04=pd.read_csv("MOZ03027F04_protozoa.csv", header=None)
MOZ03027F04.columns=["Parasites","MOZ03027F04"]
MOZ03027F04=MOZ03039E11.set_index('Parasites')

outer_merged = pd.merge(
 MOZ03027F04, MOZ03039E11, how="outer", on =["Parasites"])

This gives me the result I need, which is to make the second column to each file into a column, combining the first column as index. This is how I tried to loop:

import os
import glob
import pandas as pd

fmask = '/Users/path/to/files/species/*csv'

 dfs = []
 for f in glob.glob(fmask):
     df = pd.read_csv(f, sep=',')
     df.columns=["Parasites",f]
       df2=df.set_index('Parasites')
     df3=pd.merge(
     dfs, df2, how="outer", on =["Parasites"])

But it keeps giving me the "EmptyDataError: No columns to parse from file". All files are simple 2-column files, with names on column 1 and numbers on column 2. What am I doing wrong here?

0 Answers
Related