I am trying to find maximum similarity between two words in pandas dataframe. Here is my routine
import pandas as pd
from nltk.corpus import wordnet
import itertools
df = pd.DataFrame({'word_1':['desk', 'lamp', 'read'], 'word_2':['call','game','cook']})
def max_similarity(row):
word_1 = row['word_1']
word_2 = row['word_2']
ret_val = max([(wordnet.wup_similarity(syn_1, syn_2) or 0) for
syn_1, syn_2 in itertools.product(wordnet.synsets(word_1), wordnet.synsets(word_2))])
return ret_val
df['result'] = df.apply(lambda x: max_similarity(x), axis= 1)
It works fine, but it is too slow. I am looking for a way to speed it up. wordnet takes majority of time Any suggestions? Cython? I open to using other packages such as spacy.