Error loading trained en_core_web_trf spacyV3 NER model

user_14692044 0
Viewed 1609

After loading pretrained spacy model for finetuning on custom data

spacy.require_gpu()
nlp = spacy.load("en_core_web_trf",exclude=['tagger', 'parser', 'attribute_ruler', 'lemmatizer'])

getting error while loading for validation

model=spacy.load(category_output_dir + "/%s" % itn,exclude=['tagger', 'parser', 'attribute_ruler', 'lemmatizer'])

File "/usr/local/lib/python3.6/dist-packages/spacy/init.py", line 47, in load return util.load_model(name, disable=disable, exclude=exclude, config=config) File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 275, in load_model return load_model_from_path(Path(name), **kwargs) File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 341, in load_model_from_path return nlp.from_disk(model_path, exclude=exclude) File "/usr/local/lib/python3.6/dist-packages/spacy/language.py", line 1705, in from_disk util.from_disk(path, deserializers, exclude) File "/usr/local/lib/python3.6/dist-packages/spacy/util.py", line 1085, in from_disk reader(path / key) File "/usr/local/lib/python3.6/dist-packages/spacy/language.py", line 1700, in p, exclude=["vocab"] File "spacy/pipeline/transition_parser.pyx", line 479, in spacy.pipeline.transition_parser.Parser.from_disk File "/usr/local/lib/python3.6/dist-packages/thinc/model.py", line 529, in from_bytes return self.from_dict(msg) File "/usr/local/lib/python3.6/dist-packages/thinc/model.py", line 552, in from_dict node.set_dim(dim, value) File "/usr/local/lib/python3.6/dist-packages/thinc/model.py", line 188, in set_dim raise ValueError(err) ValueError: Attempt to change dimension 'nO' for model 'precomputable_affine' from 78 to 74

here is the code snippet

import random
from tqdm import tqdm
import spacy
import os
import copy
from spacy.tokens import Doc
from spacy.training import Example
from spacy.util import minibatch, compounding, decaying

spacy.require_gpu()
nlp = spacy.load("en_core_web_trf",exclude=['tagger', 'parser', 
'attribute_ruler', 'lemmatizer'])

ner = nlp.get_pipe('ner')
for _, annotations in TRAIN_DATA:
    for ent in annotations.get('entities'):
        ner.add_label(ent[2])

for itn in range(n_iter):
    random.shuffle(TRAIN_DATA)
    losses = {}

for batch in tqdm(minibatch(TRAIN_DATA,size=compounding(4., 64., 1.01))):
    
    for text, annots in batch:
         examples = []
         try:
              examples.append(Example.from_dict(nlp.make_doc(text), annots))
              losses=nlp.update(examples,sgd=optimizer,losses=losses)
            except:
                continue
if use_optimizer_averages:
     with nlp.use_params(optimizer.averages):
         os.mkdir(output_dir + "/%s" % itn)
         nlp.to_disk(output_dir + "/%s" % itn)
         model=spacy.load(output_dir + "/%s" % itn,exclude=['tagger', 'parser', 'attribute_ruler', 'lemmatizer'])

Train data format: TRAIN_DATA = [["Who is Shaka Khan?", {"entities": [(7, 17, "FRIENDS")]}],["I like London.", {"entities": [(7, 13, "LOC")]}]]

spacy nlp
1 Answers

It looks like you ran into a bug with release candidate v3.0.0rc2 of spaCy 3. We're working on getting this fixed for the final 3.0 release, see here: https://github.com/explosion/spaCy/pull/6460

Meanwhile, I'm afraid you won't be able to add additional labels to a transformer-based, pretrained NER model. What I would recommend though, is that you train a new NER component from scratch.

In spaCy 3, we recommend using the config system instead of writing your own training loop (https://nightly.spacy.io/usage/training#config). In a config file, you can source components from other pipelines like so:

[components.transformer]
source = "en_core_web_trf"
component = "transformer"

This allows you to reuse the transformer weights of your pretrained model, and create a new NER model on top of that (https://nightly.spacy.io/usage/embeddings-transformers):

[components.ner]
factory = "ner"

[nlp.pipeline.ner.model]
@architectures = "spacy.TransitionBasedParser.v1"
state_type = "ner"
extra_state_tokens = false
hidden_width = 128
maxout_pieces = 3
use_upper = false

[nlp.pipeline.ner.model.tok2vec]
@architectures = "spacy-transformers.TransformerListener.v1"
grad_factor = 1.0

[nlp.pipeline.ner.model.tok2vec.pooling]
@layers = "reduce_mean.v1"

You could even source the old NER component from the en_core_web_trf too, just name it differently, and you'll have two NER's running in your pipeline, each for a distinct set of labels.

Alternatively, you can also use a non-transformer-based model such as en_core_web_lg. Adding custom labels to such a pretrained model should work fine, as before.

user_79618600
Related