Spacy models with different word2vec embeddings have same performance metrics

I'm trying to compare the benefits of word vectors trained on in-domain data to word vectors trained on non-domain specific data on spaCy NER.

I have built two word2vec models, each one trained on different text.

Then, I try to build and evaluate two spaCy models, each with a different txt file containing the word2vec embeddings.

This is the code that I use to initiate the model:

!python -m spacy init vectors en models/w2v/merged/word2vec_merged_w2v.txt models/w2v/merged --name en_test

It runs succesfully, and both create different amounts of vectors.

1st model:

Creating blank nlp object for language 'en'
[+] Successfully converted 28093 vectors
[+] Saved nlp object with vectors to output directory. You can now use the path
to it in your config as the 'vectors' setting in [initialize].
C:\Users\Ruben Weijers\models\w2v\merged

2nd model:

Creating blank nlp object for language 'en'
[+] Successfully converted 34712 vectors
[+] Saved nlp object with vectors to output directory. You can now use the path
to it in your config as the 'vectors' setting in [initialize].
C:\Users\Ruben Weijers\models\w2v\quine

The vectors are different. I then try to load and train using the following code:

nlp = spacy.load("models/w2v/merged")
nlp.add_pipe('ner')
nlp.to_disk("models/w2v/merged")
#train model
#train model
!python -m spacy train models/w2v/merged/config.cfg --output models/w2v/merged --paths.train data/train2.spacy --paths.dev data/test2.spacy --paths.vectors models/w2v/merged

I do the same for the other model, pathing to the other vector file ofcourse.

However, the training pipeline shows that both models have exactly the same precision, recall and loss rate throughout the learning process.

Also, when I call:

#evaluate model
!python -m spacy evaluate models/w2v/merged/model-best data/val2.spacy --output models/w2v/merged/metrics.json

Both models have the same performance metrics, while the vectors are totally different.

I have looked up different videos on how to path vectors, I have added paths to the vectors to the config files, on top of adding

--paths.vectors models/w2v/merged

All doesn't seem to help. Many videos show how to implement word2vec, yet don't evaluate. I'm curious to see as to why both word2vec models appear to be exactly the same. It doesn't make sense. I have checked that pathing is correct, and files are in the correct place multiple times. It doesn't seem like that's the issue since the different numbers returned in creation of vectors also shows that the vector files are different.

I have created the word vectors using:

def train_w2v_model(model_name):
    w2v_model = Word2Vec(min_count=5,
                        window=2,
                        vector_size=500,
                        sample=6e-5,
                        alpha=0.03,
                        min_alpha=0.0007,
                        negative=20)
    w2v_model.build_vocab(sentences)
    w2v_model.train(sentences, total_examples=w2v_model.corpus_count, epochs=30)
    w2v_model.save(f"downloads/{model_name}.model")
    w2v_model.wv.save_word2vec_format(f"downloads/word2vec_{model_name}.txt")