Is there any way to convert python pandas dataframe to NLP corpus or documentation?

Viewed 1004
1 Answers

Any list of list of words can be converted to list of TaggedDocument & used to train a doc2vec model. OR you can split a dataframe's text column into list of words first and then iterate over output pandas series to create a list of TaggedDocument. Here is example usage for your dataset.Lets assume you load you pandas dataframe as df, then

output_ser = df.loc[:, 'Mobile Phase'].str.split(r'[/\s\\]+')
documents = [TaggedDocument(doc, [i]) for i, doc in enumerate(output_ser)]
model = Doc2Vec(documents, vector_size=5, window=2, min_count=1, workers=1)

Follow this link https://radimrehurek.com/gensim/models/doc2vec.html for usage example

Related