Sorry if this has been answered somewhere, but I was unable to find any examples where the person asking was actually using a list rather than a string. I totally might just be doing something dumb here, but I cannot seem to figure it out. This image shows my code - I am simply trying to initialize a dictionary in gensim using a list, but I am getting an error message saying that I am passing in a string rather than a list. I tried printing the type as well as the list itself to make sure it was in fact a list, and as you can see, it is. I also tried removing any tokens containing apostrophes in case that was for some reason causing my issues, but to no avail. I have been staring at my code forever and cannot figure out what it is so upset about. Sorry again if I am just missing something stupid, and thanks for any help you are able to provide! :)
Edit: also here is my full error message if that helps.
Edit #2: Here is my code (a bit different than before, just tried removing the punctuation).:
import gensim.corpora as corpora
import regex as re
print(type(verbs))
print(verbs)
nopunc = []
for word in verbs:
if not re.compile(".*\'.*").match(word):
nopunc.append(word)
print(nopunc)
id2word = corpora.Dictionary(nopunc)
Here is my error message:
Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_35400\185953347.py in <module>
13 print(nopunc)
14
---> 15 id2word = corpora.Dictionary(nopunc)
16 texts = nopunc
17
~\AppData\Roaming\Python\Python37\site-packages\gensim\corpora\dictionary.py in __init__(self, documents, prune_at)
76
77 if documents is not None:
---> 78 self.add_documents(documents, prune_at=prune_at)
79 self.add_lifecycle_event(
80 "created",
~\AppData\Roaming\Python\Python37\site-packages\gensim\corpora\dictionary.py in add_documents(self, documents, prune_at)
202
203 # update Dictionary with the document
--> 204 self.doc2bow(document, allow_update=True) # ignore the result, here we only care about updating token ids
205
206 logger.info("built %s from %i documents (total %i corpus positions)", self, self.num_docs, self.num_pos)
~\AppData\Roaming\Python\Python37\site-packages\gensim\corpora\dictionary.py in doc2bow(self, document, allow_update, return_missing)
239 """
...
--> 241 raise TypeError("doc2bow expects an array of unicode tokens on input, not a single string")
242
243 # Construct (word, frequency) mapping.
TypeError: doc2bow expects an array of unicode tokens on input, not a single string