I'm attempting to update a pre-trained spacy model en_core_web_md with a few rounds of a beam objective other than beam_width = 1, and I cannot seem to find the right way to pass the different parameters into the **cfg such that the model uses them for training (at THIS point).
This was my latest attempt:
pipe_exceptions = ["ner", "trf_wordpiecer", "trf_tok2vec"]
other_pipes = [pipe for pipe in nlp.pipe_names if pipe not in pipe_exceptions]
# only train NER
with nlp.disable_pipes(*other_pipes), warnings.catch_warnings():
# show warnings for misaligned entity spans once
warnings.filterwarnings("once", category=UserWarning, module='spacy')
# TRY TO FORCE BEAM TRAINING INSTEAD OF GREEDY METHOD
nlp.use_params({'ner':{'beam_width':16, 'beam_density':0.0001}})
print(nlp.meta)
sizes = compounding(1.0, 4.0, 1.001)
# batch up the examples using spaCy's minibatch
for itn in range(n_iter):
random.shuffle(TRAIN_DATA_2)
batches = minibatch(TRAIN_DATA_2, size=sizes)
losses = {}
for batch in batches:
texts, annotations = zip(*batch)
nlp.update(texts,
annotations,
sgd=optimizer,
drop=0.35,
losses=losses
)
print("Losses", losses)
However, after training, the model/ner/cfg file still lists:
{
"beam_width":1,
"beam_density":0.0,
"beam_update_prob":1.0,
...
So, I have a few questions:
- Am I able to update an existing greedy trained model with a new beam objective?
- If true, how can I properly changed these training parameters (and confirm they are changed)?
- If false, for a new from scratch model, how can I properly changed these training parameters (and confirm they are changed)?
Why do this? I am attempting to train a model that provides probabilities for NER decisions that I can surface to my users. THIS post and a few others show how to use beam_parse to obtain probabilities after the fact from a greedy model. However, they all mention that the greedy model hasn't been trained with a global objective, so these scores aren't especially meaningful unless you also perform some iterations of beam training as well. (link to github issue)