I used deeplearning4j library to do vectorization task using word2vec. I need to vectorize new word after training the model using a specific corpus. so how to add this new word and update the training to get a new weight vector for the new word? my code as folows:
Test word2Vec = new Test();
word2Vec.train();
//test the generated trained file
Word2Vec word2VecModel = WordVectorSerializer.readWord2VecModel(new File(word2Vec.modelFilePath));
double cosi=word2VecModel.similarity("httpdbpediaorgresourcethe_terminator", "httpdbpediaorgresourceterminator_salvation");
System.out.println(cosi);
}
public void train() throws IOException {
SentenceIterator sentenceIterator = new FileSentenceIterator(new File(inputFilePath));
TokenizerFactory tokenizerFactory = new DefaultTokenizerFactory();
tokenizerFactory.setTokenPreProcessor(new CommonPreprocessor());
Word2Vec vec = new Word2Vec.Builder()
.layerSize(100)
.windowSize(5)
.epochs(5) //3-50 https://arxiv.org/pdf/1301.3781.pdf
.elementsLearningAlgorithm(new SkipGram<VocabWord>())
.iterate(sentenceIterator)
.tokenizerFactory(tokenizerFactory)
.build();
vec.fit();
WordVectorSerializer.writeWordVectors(vec, modelFilePath);
}