Pandas apply two argument with Spacy

user_15112773 0
Viewed 115
 df.head()
        lang    text
    0  eng      Johnnet went out on the field and felt under her feet..
    1  eng      John was shocked by this statement..
    2  de        Heute hat Marie köstlich gegessen und..

I have a dataframe with different languages that why i have a dictionary of two languages spacy :

eng_nlp= spacy.load('en_core_web_lg')
de_nlp= spacy.load('de_core_news_lg')

spacy_lang = {
    'de': de_nlp,
    'eng': eng_nlp
}

I wrote a function that looks displays only people in the column depending on the language.

def label_lang(lang,text):
    model = spacy_lang[lang]
    doc = model(text)
    for ent in  doc.ents:
         if  (ent.label_ == 'PERSON'):
                 return ent.text

NOW I WANT TO APPLY THIS TO THE COLUMN df['text'], BUT I GET AN ERROR

df.apply( lambda x: label_lang(spacy_lang[x],x['text']),axis = 1)

TypeError: unhashable type: 'Series'

I don't understand what I should use as an argument function (spacy_lang)

spacy apply label python
1 Answers

x is a series, not a string, thus you can't use spacy_lang[x] as the dictionary key type expected here is a string. In this case, you need to use spacy_lang[x['lang']] instead of spacy_lang[x] in the lambda.

More, if you get the model in the lambda, you need not re-try getting the model inside the label_lang function. Look, you have label_lang(spacy_lang[x].... and model = spacy_lang[lang], where the latter already holds the spacy model in the lang variable.

You can use

import spacy
import pandas as pd

eng_nlp= spacy.load('en_core_web_lg')
de_nlp= spacy.load('de_core_news_lg')
spacy_lang = {'de': de_nlp,'eng': eng_nlp}

def label_lang(model,text):
    doc = model(text)
    for ent in  doc.ents:
         if  (ent.label_ == 'PERSON'):
                 return ent.text

df = pd.DataFrame({'lang': ['eng','eng', 'de'], 'text':[
    'Johnnet went out on the field and felt under her feet..', 
    'John was shocked by this statement..',
    'Heute hat Marie köstlich gegessen und']})

print(df.apply( lambda x: label_lang(spacy_lang[x['lang']],x['text']),axis = 1))

Output:

0    None
1    John
2    None
user_38329700
Related