Number is recognized as a noun in spacy portuguese model

Viewed 42

Just out of curiosity I would like to ask why the number "4950" has the PoS (part of speech) of "NOUN" in spaCy v3.1.3, using the large model in Portuguese. It is not in the GitHub token exception file (https://github.com/explosion/spaCy/blob/master/spacy/lang/pt/tokenizer_exceptions.py).

nlp = spacy.load('pt_core_news_lg')
doc = nlp('4950')
print(doc[0].text, doc[0].pos_)

#4950 NOUN

Is there any way to know what the other particular cases are?

1 Answers

To be clear, this should normally be a NUM.

This looks like it's just an error, and it doesn't affect most numbers, including similar ones like 4951. It's possible that somewhere in the Portuguese training data 4950 is labelled NOUN for some reason.

It's hard to explain individual predictions by the statistical models, and they make errors sometimes. This one is particularly egregious and may indicate an issue with data preparation, but in general errors like this are always possible. See this thread.

Also note this doesn't seem to be an issue in the small model. I'll look into this internally to see if there's a bug somewhere.


Quick update: If you use this in a sentence, like 4950 maçãs, it's properly labelled as NUM. One-word sentences are not something the models are trained on a lot and might cause more weird results.

Related