I'm trying to run the distributed LDA example as described here:
https://radimrehurek.com/gensim/dist_lda.html
I created the set of documents by following the tutorial here:
https://radimrehurek.com/gensim/dist_lsi.html
by "inflat[ing] the corpus to 1M documents, by repeating its documents over&over" as it suggests
I'm using python 3.3 and numpy 1.9.2 I keep getting the following error:
Exception in thread oneway-call:
Traceback (most recent call last):
File "/usr/lib64/python3.3/threading.py", line 901, in _bootstrap_inner
self.run()
File "/usr/lib/python3.3/site-packages/Pyro4/core.py", line 1484, in run
super(_OnewayCallThread, self).run()
File "/usr/lib64/python3.3/threading.py", line 858, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib64/python3.3/site-packages/gensim/models/lda_worker.py", line 71, in requestjob
self.processjob(job)
File "/usr/lib64/python3.3/site-packages/gensim/utils.py", line 98, in _synchronizer
result = func(self, *args, **kwargs)
File "/usr/lib64/python3.3/site-packages/gensim/models/lda_worker.py", line 80, in processjob
self.model.do_estep(job)
File "/usr/lib64/python3.3/site-packages/gensim/models/ldamodel.py", line 480, in do_estep
gamma, sstats = self.inference(chunk, collect_sstats=True)
File "/usr/lib64/python3.3/site-packages/gensim/models/ldamodel.py", line 423, in inference
if doc and not isinstance(doc[0][0], six.integer_types):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I ran the distributed lsi example and it ran fine, but for some reason I couldn't seem to get the lda to work.
I tried changing line 423 in /usr/lib64/python3.3/site-packages/gensim/models/ldamodel.py to:
if doc is not None and not isinstance(doc[0][0], six.integer_types):
The error went away, but I got a warning that
FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
Could someone explain what I've done wrong? Is my change to this file correct? Or should I be running LDA differently?