Spacy 1 vs spacy 2 (spacy-nightly) Have they changed data-model? Why similarity calculation does not work?

user_8517006 0
Viewed 477

I understand that spacy 2 alpha (or called spacy-nightly) is building vectors of words based on their context - so I do understand differences between values of similarity for words in nlp('apples oranges') and separated nlp('apples') and nlp('oranges') (and of course I am using different models for spacy 1 and spacy 2). But what I do not understand how I am suppose to call/input strings let say into similarity method.

Do they changed data model? I have not found anything in documentation... Am I doing something wrong - i.e. are my results reasonable?

Of course I am running those codes in separated virtual environments

spacy 1 - this works fine:

print(nlp('apples').similarity(nlp('oranges')))
# 0.77809414836

spacy 2 - this returns 0.0 - so it does not work:

print(nlp('apples').similarity(nlp('oranges')))
# 0.0

Any ideas? Code below shows what works for me and what does not..

import spacy # version spacy (1.9.0)
nlp = spacy.load('en_core_web_md') 

doc = nlp('apples oranges')
print(doc[0].similarity(doc[1]))
#0.77809414836
print(nlp('apples')[0].similarity(nlp('oranges')[0]))
#0.77809414836
print(nlp('apples').similarity(nlp('oranges')))
# 0.77809414836

#----------------

import spacy # spacy-nightly (2.0.0a14)
nlp = spacy.load('en_core_web_sm')

doc = nlp('apples oranges')
print(doc[0].similarity(doc[1]))
# 0.630915
print(nlp('apples')[0].similarity(nlp('oranges')[0]))
# 0.892392
print(nlp('apples').similarity(nlp('oranges')))
# 0.0
word2vec spacy nlp python
1 Answers

Well here it goes an answer with super delay.

The short answer is Yes, Spacy v2 introduces a lot of changes, and brand new models so you might experience some weird situations in case you don't update the models too and re-train your own models.

A brief quote from release page at Github:

Note that the old v1.x models are not compatible with spaCy v2.0.0. If you've trained your own models, you'll have to re-train them to be able to use them with the new version. For a full overview of changes in v2.0, see the documentation and guide on migrating from spaCy 1.x.

For you and all the guys who have to migrate from old Spacy v1 to v2 (still old since now we have v3) I would recommend that you first:

  1. Read this amazing migration guide from v1 to v2
  2. It might also help to read what's new in v2 here

Hope this answer helps some users who end up on this post like me, also due to issues with spacy vectors after upgrading Spacy version.

user_79931630
Related