How do I make this function for concatenating Excel sheets from a single file more efficient?

Viewed 634

I have several Excel files from which I would like to generate individual dataframes, which will be concatenations of an arbitrary subset of sheets within each file.

It's important that I be able to subset the sheets (here I do that by indexing in to the list of sheet_names), and also in some manner track provenance (as I do here with the assign method).

This code currently works, but I'm concerned that I'm missing some fundamental way to make it more efficient (for when I wind up with a file with, say, 20 sheets).

I've already reviewed several other questions1 pertaining to concatenating, say, multiple CSVs or Excel files, but have a hard time generalizing those to the problem of parsing sheets within one file, with respect to efficiency.

This question is partially driven by my having used generators to concatenate Excel files before, but I'm having a hard time generalizing those cases to this one.

Example Excel File on GitHub

import pandas as pd
import xlrd

import xlwt

def file_parser(file):
    df_list = []
    for x in file.sheet_names[1::]:
        df = file.parse(x).assign(Source=x)
        df_list.append(df)
    return df_list

1 How to parse dataframes from an excel sheet with many tables (using Python, possibly Pandas)

   Import multiple csv files into pandas and concatenate into one DataFrame

   Pands ExcelFile.parse() reading file in as dict instead of dataframe

1 Answers
Related