I am trying to use Gensim in a AWS Glue job. I have added the gensim wheel file in the python library path. Upon job execution, I get the following error stating the Cython extension is unavailable and thus the gensim modules are unusable. I have downloaded the gensim wheel file from IFD UCI edu website here. Do I need to rebuild the gensim wheel to resolve the cython error? If yes, please advise how to include the cython extensions.
Python version in Glue env: 3.6
Gensim wheel: gensim-3.8.3-cp36-cp36m-win_amd64.whl
Error
Traceback (most recent call last):
File "/glue/lib/installation/gensim/matutils.py", line 1426, in <module>
from gensim.corpora._mmreader import MmReader # noqa: F401
ModuleNotFoundError: No module named 'gensim.corpora._mmreader'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/runscript.py", line 211, in <module>
runpy.run_path(temp_file_path, run_name='__main__')
File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/tmp/glue-python-scripts-1qvmz94x/dmp-gcaf-slr-train-lda-model.py", line 11, in <module>
File "/glue/lib/installation/gensim/__init__.py", line 5, in <module>
from gensim import parsing, corpora, matutils, interfaces, models, similarities, summarization, utils # noqa:F401
File "/glue/lib/installation/gensim/corpora/__init__.py", line 6, in <module>
from .indexedcorpus import IndexedCorpus # noqa:F401 must appear before the other classes
File "/glue/lib/installation/gensim/corpora/indexedcorpus.py", line 15, in <module>
from gensim import interfaces, utils
File "/glue/lib/installation/gensim/interfaces.py", line 21, in <module>
from gensim import utils, matutils
File "/glue/lib/installation/gensim/matutils.py", line 1428, in <module>
raise utils.NO_CYTHON
RuntimeError: Cython extensions are unavailable. Without them, this gensim functionality is disabled. If you've installed from a package, ask the package maintainer to include Cython extensions. If you're building gensim from source yourself, run `python setup.py build_ext --inplace` and retry.