Extracting all nlp entities to another column

Question

I'm trying to extract all entities to another column in my dataset.

I've tried this code below on a small portion of my full dataset and it works, but once I put in my whole dataset I get the following error:

TypeError: object of type 'float' has no len()

def all_ents(v):
    return [(ent.text, ent.label_) for ent in nlp(v).ents]

dt['Entities'] = dt['text'].apply(lambda v: all_ents(v))

Can anyone help? Thanks in advance!

user_3832970 · Answer

Your text column contains some float values or np.nans.

It is safer to cast all the values of that column to a string:

dt['Entities'] = dt['text'].astype(str).apply(lambda v: all_ents(v))
#                          ^^^^^^^^^^^^