Best way to detect person name / entity name from French text

user_11803187 0
Viewed 1477

I am trying to detect person name or company/institute name from French texts. I have tried the following and the results are not good at all.

import spacy

# or any of the other two models fr_core_news_sm, fr_core_news_lg
nlp = spacy.load("fr_core_news_md")  

text ="""Tous les vents,
Balayent les mots de coeur
Moi, j'suis comme le vent:
L'esprit à mille à l'heure,
Je juge sans doute trop vite
C'est ok, tant pis
C'est juste là, je m'agite
Je grandis, l'amour aussi
C'est au gré du vent
Que j'aime vagabonder,
Moi, je suis comme le vent
J'embrasse toute une armée
De rêves et de bleuets,
Me plonger dedans
Je sais ce que je sais 
Rapport de: Andre STE-GERMAINE,
"""

doc_fr = nlp_fr(text_fr)

Well, it marked

  1. "Balayent as PERSON which should not,
  2. Rapport de: Andre STE-GERMAINE', 'MISC' which should be PERSON

I am not sure if NLTK can help on this.

Is there another better tool to help with this task?

Thanks.

spacy stanford-nlp nltk nlp python-3.x
1 Answers

The model you have chosen is trained on selected data that might not represent your own dataset. You could finetune the model by supplying more annotated data to create a better overlap.

Reformatting the text data should also increase the performance of the model as the text supplied is a little bit hard to understand by a Spacy model. Sentences are broken into pieces instead of full readable text, words are capitalized even though they shouldn't be.

I don't speak French, but have reformatted your text a little bit and seem already to get better results.

import spacy
nlp = spacy.load("fr_core_news_md")
text ="""Tous les vents, balayent les mots de coeur.
Moi, j'suis comme le vent: L'esprit à mille à l'heure, je juge sans doute trop vite.
C'est ok, tant pis.
C'est juste là, je m'agite.
Je grandis, l'amour aussi.
C'est au gré du vent.
Que j'aime vagabonder, moi, je suis comme le vent.
J'embrasse toute une armée de rêves et de bleuets,
Me plonger dedans je sais ce que je sais.
Rapport de: Andre STE-GERMAINE,
"""
doc = nlp(text)
for ent in doc.ents:
    print(ent.text, '-',ent.label_)

This results in:

Andre STE - PER

I explain in the comments to this post why the Person isn't fully tagged, but let's do it here again. The last sentence of this text isn't really a sentence on which the Spacy NER can really perform accurate as it isn't a real sentence which has been trained on. The name is also fully capitalized, which is not normal for anybody's surname which also confused the algorithm even more.

In conclusion, supply it with data that it is trained on, reformat your text, this would increase the performance significantly.

user_67011630
Related