NLP - speed up word similarity matchin

Viewed 2200

I am trying to find maximum similarity between two words in pandas dataframe. Here is my routine

import pandas as pd
from nltk.corpus import wordnet
import itertools

df = pd.DataFrame({'word_1':['desk', 'lamp', 'read'], 'word_2':['call','game','cook']})

def max_similarity(row):
    word_1 = row['word_1']
    word_2 = row['word_2']

    ret_val = max([(wordnet.wup_similarity(syn_1, syn_2) or 0) for 
       syn_1, syn_2 in itertools.product(wordnet.synsets(word_1), wordnet.synsets(word_2))])

    return ret_val

df['result'] = df.apply(lambda x: max_similarity(x), axis= 1)

It works fine, but it is too slow. I am looking for a way to speed it up. wordnet takes majority of time Any suggestions? Cython? I open to using other packages such as spacy.

2 Answers
Related