I am trying to use Gensim with Glove instead of word2vec. To make the shape of Glove compatible with Gensim and use it, I am using the following lines of code:
import gensim
from gensim.scripts.glove2word2vec import glove2word2vec
glove_in = 'glove.840B.300d.txt'
word2vec_format_out = 'glove.840B.300d.txt.word2vec'
glove2word2vec(glove_in, word2vec_format_out)
model =
gensim.models.KeyedVectors.load_word2vec_format(word2vec_format_out,
encoding='utf-8', binary=True)
However, this last line of code gives the following error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbd in position 0:
invalid start byte
I have tried to open Glove first and then writing as a csv file, then re-open specifying encoding='utf-8'. I also tried several other things mentioned here, but the error keeps coming back. Does anyone know a solution for this?