How to concatenate efficiently a list of dataframes?

Question

I have a function crawl that input an integer and output a dataframe. To concatenate dataframes df1, df2, df3, we can use pandas.concat([df1, df2, df3]). I would like to ask of there is an efficient way to concatenate df1,..., df17 without writing a long list df1 = crawl(1),..., df17 = crawl(17).

Thank you so much!

user_4238408 · Answer

How about:

pd.concat([crawl(i) for i in range(1,18)])

user_5820814 · Answer

You can do this:

df_list = [crawl(i) for i in range(1,18)]

final_df = pd.concat(df_list)

How to concatenate efficiently a list of dataframes?

2 Answers