How to read all csv files in a folder in pandas?

user_14758478 0
Viewed 14308

Every month I am downloading the whole ledger in csv format and saving the same to a folder. How I can read it in pandas as single dataframe

pandas directory
3 Answers

fthomson's answer but the last line should be

import pandas as pd
import glob
files = glob.glob("path/*.csv")

df = pd.DataFrame()
for f in files:
    csv = pd.read_csv(f)
    df = df.append(csv)
user_122892440

This is a faster way. Append small DataFrame after large DataFrame will cost a lot. So a better way is to append all DataFrame into one list and use pd.concat to concat all DataFrame.

import pandas as pd
import glob
files = glob.glob("path/*.csv")
df = []
for f in files:
    csv = pd.read_csv(f)
    df.append(csv)
df = pd.concat(df)
user_179292730

I'm doing this from memory but this is the general idea

import pandas as pd
import glob
files = glob.glob("path/*.csv")

df = pd.DataFrame()
for f in files:
    csv = pd.read_csv(f)
    df = df.append(csv)
user_145869670
Related