Difference between spaCy's (v3.0) `nlp.make_doc(text)` and `nlp(text)`? Why should we use `nlp.make_doc(text)` when training?

user_9436980 0
Viewed 2123

I understand that we should create Example objects and pass it to the nlp.update() method. According to the example in the docs, we have

for raw_text, entity_offsets in train_data:
    doc = nlp.make_doc(raw_text)
    example = Example.from_dict(doc, {"entities": entity_offsets})
    nlp.update([example], sgd=optimizer)

And looking at the source code of the make_doc() method, it seems like we would be just tokenizing the input text and then annotating the tokens.

But the Example object should have the reference/"gold-standard" and the predicted values. How does the information ends up in the document when we call nlp.make_doc()?

Additionally, when trying to get the predicted entity tags (using a trained nlp pipeline) back from the Example object I get no entities (though I could if I had created the object with nlp(text). And training crashes if I try using nlp(text) instead of nlp.make_doc(text) with

...
>>> spacy.pipeline._parser_internals.ner.BiluoPushDown.set_costs()
ValueError()
named-entity-recognition spacy python
1 Answers

You can feel free to ask this sort of question on the Github Discussions board as well. Thanks also for taking time to think about this and read some of the code before asking. I wish every question were like this.

Anyway. I think the Example.from_dict() constructor might be getting in the way of understanding how the class works. Does this make things clearer for you?

from spacy.tokens import Doc, Span
from spacy.training import Example
import spacy
nlp = spacy.blank("en")

# Build a reference Doc object, representing the gold standard.
y = Doc(
    nlp.vocab,
    words=["I", "work", "at", "Berlin!", ".", "It", "'s", "a", "hipster", "bar", "."]
)
# There are other ways we could set up the Doc object, including just passing
# stuff into the constructor. I wanted to show modifying the Doc to set annotations.
ent_start = y.text.index("Berlin!")
assert ent_start != -1
ent_end = ent_start + len("Berlin!")
y.ents = [y.char_span(ent_start, ent_end, label="ORG")]
# Okay, so we have our gold-standard, aka reference aka y, Doc object.
# Now, at runtime we won't necessarily be tokenizing that input text that way.
# It's a weird entity. If we only learn from the gold tokens, we can never learn
# to tag this correctly, no matter how many examples we see, if the predicted tokens
# don't match this tokenization. Because we'll always be learning from "Berlin!" but
# seeing "Berlin", "!" at runtime. We'll have train/test skew. Since spaCy cares how
# it does on actual text, not just on the benchmark (which is usually run with 
# gold tokens), we want to train from samples that have the runtime tokenization. So
# the Example object holds a pair (x, y), where the x is the input.
x = nlp.make_doc(y.text)
example = Example(x, y)
# Show the aligned gold-standard NER tags. These should have the entity as B-ORG L-ORG.
print(example.get_aligned_ner())

The other piece of information that might explain this is that the pipeline components try to deal with partial annotations, so that you can have rules which are presetting some entities. This is what's happening when you have a fully annotated Doc as the x --- it's taking those annotations as part of the input, and there's no valid action for the model when it tries to construct the best sequence of actions to learn from. The usability for this situation could be improved.

user_2321190
Related