Spacy Dutch noun_phrases returns empty list using nl_core_news_sm

Viewed 221

I want to extract the noun_phrases of a Dutch text using the model nl_core_news_sm by spacy. It returns an empty list On the other hand the equivalent English model en_core_web_sm provides indeed the list of noun_chunks (noun_phrases)

Is this normal behaiviour? i.e. the Dutch language moedel does not include the noun_phrases separator and the english model does? Or am I doing something wrong?

string='''In een wereld waarin je wordt overspoeld met informatie, is het prettig om een nieuwsbron te hebben met heldere stukken, die de ruimte laten om je eigen mening te vormen.'''
nlp = spacy.load('nl_core_news_sm')
print(dir(doc))
print(doc.noun_chunks)
list_chunks=[chunk for chunk in doc.noun_chunks]
for chunk in doc.noun_chunks:
    print(chunk.text)

The result here is that list_chunks is [] And of course nothing is printed in the loop

I used dir(doc) to compare the methods available in order to compare to the English model. They are the same.

nlp_en = spacy.load('en_core_web_sm')
string='''They normally organises a wide range of activities for kids in the summer holidays. Due to the virus, these have all been cancelled'''
doc2=nlp_en(string)
print(dir(doc2))
print(doc2.noun_chunks)
for chunk in doc2.noun_chunks:
    print(chunk.text)

In English it works.

Some idea?

EDIT NOTE: Here I compared three language models: enter image description here

1 Answers

I can tell you that the noun chuck tokenizer is simply not yet implemented in that Dutch language model. It's not a bug. It simply has to be done. So whoever reads this please check the date of the question. Obviously the spacy guys will do it but it might take a while.

Related