FastText top10 similar words

user_18002337 0
Viewed 30

I am working in an NLP task using the following FastText model,

# FastText
ft_model = FastText(word_tokenized_corpus,
                    max_n=0,
                    vector_size=64,
                    window=5,
                    min_count=1,
                    sg=1,
                    workers=20,
                    epochs=1,
                    seed=42)

I have the following code to see the top10 similar words for each word

random_words = ['ready', 'stopping', 'myself', 'follow', 'instagram',
                'people', 'stories', 'coffee', 'hang', 'tumblr', 'snapchat']

full_semantically_similar_words_wv_2 = {words: [item[0] for item in ft_model.wv.most_similar([words], topn=10)]
                  for words in random_words}

for k,v in full_semantically_similar_words_wv_2.items():
    print(k + ": " + str(v))

The output looks like this

ready: ['prepared', 'ready.', 'gonna', 'gunna', 'going', 'guaranteed', 'ganna', 'waiting', 'preparing', 'gona']
stopping: ['preventing', 'noticing', 'disappearing', 'calling.', 'resorting', 'slowing', 'stoping', 'distracting', 'sane', 'pushing']
myself: ['myself.', 'myself...', 'myself..', 'myself....', 'myseld', 'myslef', 'them....', 'it.....', 'myself-', 'them']
follow: ['unfollow', 'unlisten', 'reblog', 'stalk', 'block', 'softblock', 'dm', 'follow.', 'join', 'accordingly']
instagram: ['insta', 'twitter', 'facebook', 'tumblr', 'snapchat', 'musical.ly', 'instagram.', 'fb', 'twitter.', 'insta.']
people: ['people.', 'people..', 'ppl', 'people...', 'people....', 'ppl.', 'people-', 'strangers', 'them.', 'others']
stories: ['stories.', 'poems', 'storys', 'story', 'inspirational', 'reactions', 'characters.', 'videos.', 'animations', 'story.']
coffee: ['tea', 'coffe', 'soda.', 'iced', 'soda', 'chai', 'wine', 'coffee.', 'creamer', 'boba']
hang: ['hangout', 'invite', 'bail', 'reconnect', 'hangin', 'blurt', 'sneak', 'reunite', 'clubbing', 'arrange']
tumblr: ['twitter', 'instagram', 'musical.ly', 'pinterest', 'insta', 'ao3', 'twitter.', 'facebook', 'deviantart', 'tumblr.']
snapchat: ['instagram', 'insta', 'sc', 'facebook', 'snapchat.', 'insta.', 'instagram.', 'fb', 'twitter', 'kik.']

The output that I want to create should be a list with all the words, like

['prepared', 'ready.', 'gonna', ..., 'fb', 'twitter', 'kik.']

I woyld be grateful if you could help me.

nlp python
1 Answers

I assume full_semantically_similar_words_wv_2 is a dictionary. Then, you can add all the words to a single list and print it like this:

all_words = []
for v in full_semantically_similar_words_wv_2.values():
    all_words += v
print(all_words)

Or in one line:

all_words = [word for word_list in full_semantically_similar_words_wv_2.values() for word in word_list]
print(all_words)
user_78905610
Related