Why does Pandas read_excel function return an error in Pyinstaller .exe but not under Python interpreter?

user_8302503 0
Viewed 1351

I'm using the Pandas read_excel function to import data from a spreadsheet. This works fine when run under the Python interpreter, but when I build an exe with PyInstaller it returns an IndexError.

Here's a simplified code pandas_test.py that demonstrates the problem:

import pandas as pd

filepath = 'C:/Users/User/Documents/Development/Python/PHL/Test Data/Study 
template mock-up.xlsx'
df = pd.read_excel(filepath, sheet_name='Data Entry', index_col=9)
print(df.head())

This runs just fine under Python 3.6 with pandas 0.23.4 and xlrd 1.1.0.

When I build pandas_test.py with PyInstaller it successfully produces an pandas_test.exe, but I get this error:

Traceback (most recent call last):
  File "pandas_test.py", line 4, in <module>
  File "site-packages\pandas\io\excel.py", line 212, in read_excel
  File "site-packages\pandas\io\excel.py", line 513, in _parse_excel
  File "site-packages\pandas\io\parsers.py", line 1912, in TextParser
  File "site-packages\pandas\io\parsers.py", line 764, in __init__
  File "site-packages\pandas\io\parsers.py", line 995, in _make_engine
  File "site-packages\pandas\io\parsers.py", line 2021, in __init__
  File "site-packages\pandas\io\parsers.py", line 2772, in _get_index_name
  File "site-packages\pandas\io\parsers.py", line 3084, in _clean_index_names
IndexError: list index out of range
[17264] Failed to execute script pandas_test

I've read through the PyInstaller output, but there's nothing obviously related:

887 INFO: PyInstaller: 3.3.1
887 INFO: Python: 3.6.2
889 INFO: Platform: Windows-10-10.0.17134-SP0
892 INFO: wrote C:\Users\User\Documents\Development\Python\PandaTest\pandas_test.spec

I could understand if some module was not found, but why an IndexError only in the deployed code?

pandas pyinstaller python
2 Answers

I met similar errors as you but in my case there is no warning until I run it to get "fatal error detected" for below script (Excelfile.parse is equivalent to read_excel(ExcelFile, ...), just try to see if Pyinstaller can work with it or not)

import xlrd
import pandas as pd
from os.path import join, isfile
from os import environ
if isfile(join(environ['USERPROFILE'],'Downloads','Report_15_13__12_12_2018.xlsx')):
    rd=pd.Excelfile(join(environ['USERPROFILE'],'Downloads','Report_15_13__12_12_2018.xlsx'))
    df1=rd.parse()
    df1.to_excel(join(environ['USERPROFILE'],'Downloads','test.xls'))

I can't find answer on how to overcome but by copying code from pandas (https://github.com/pandas-dev/pandas/blob/v0.24.1/pandas/io/excel.py#L658-L718) I successfully worked around it. For my case I have only 1 sheet, but I believe multi sheets can be easily supported.

from xlrd import (xldate, XL_CELL_DATE,
                          XL_CELL_ERROR, XL_CELL_BOOLEAN,
                          XL_CELL_NUMBER,open_workbook)

from datetime import date, datetime, time, timedelta
from pandas import DataFrame
from numpy import array,nan
from os import environ
from os.path import join

book=open_workbook(join(environ['USERPROFILE'],'Downloads','excel_to_read.xls'))
epoch1904 = book.datemode
sheet=book.sheet_by_index(0)
def _parse_cell(cell_contents, cell_typ):
    """converts the contents of the cell into a pandas
        appropriate object"""

    if cell_typ == XL_CELL_DATE:

        # Use the newer xlrd datetime handling.
        try:
            cell_contents = xldate.xldate_as_datetime(
                cell_contents, epoch1904)
        except OverflowError:
            return cell_contents

        # Excel doesn't distinguish between dates and time,
        # so we treat dates on the epoch as times only.
        # Also, Excel supports 1900 and 1904 epochs.
        year = (cell_contents.timetuple())[0:3]
        if ((not epoch1904 and year == (1899, 12, 31)) or
                (epoch1904 and year == (1904, 1, 1))):
            cell_contents = time(cell_contents.hour,
                                    cell_contents.minute,
                                    cell_contents.second,
                                    cell_contents.microsecond)

    elif cell_typ == XL_CELL_ERROR:
        cell_contents = nan
    elif cell_typ == XL_CELL_BOOLEAN:
        cell_contents = bool(cell_contents)
    elif cell_typ == XL_CELL_NUMBER:
        # GH5394 - Excel 'numbers' are always floats
        # it's a minimal perf hit and less surprising
        val = int(cell_contents)
        if val == cell_contents:
            cell_contents = val
    return cell_contents

data = []


for i in range(sheet.nrows):
    row = [_parse_cell(value, typ)
            for value, typ in zip(sheet.row_values(i),
                                    sheet.row_types(i))]
    data.append(row)
NoOfColumns=len(sheet.row_values(i))
NoOfRows=sheet.nrows-1
DataFrame(array(data[1:]).reshape(NoOfRows,NoOfColumns),columns=data[0]).to_excel(join(environ['USERPROFILE'],'Desktop','test.xlsx'),index=False, engine='xlsxwriter')
user_103127380

I also got this problem. In my case, I fixed it by installing xlrd and openpyxl in the virtualenv.

user_139515260
Related