Is there an easier way to do custom lemmatization?

user_14352335 0
Viewed 41

I have a dataframe of two columns. First one["lemm"] of what words to change if they occur. Second one["word"], what to change them to. I'm new to this so I spent a lot of time writing what I think is sloppy code as it iterates through a df and is slow. But, I just want to know if there's a better way to do this for my own learning.

lemm is the df with the two columns(The Lemmatization df). df is where I need to do the lemmatization(just 1 column with a bunch of words).

for words in lemm["lemm"]:
    if words in df[0].values:
        df[df[0]==words] = lemm['word'].loc[lemm["lemm"]==words].values[0]
pandas lemmatization python
1 Answers

Here is an example to illustrate another way to do it:

import pandas as pd

lemm = pd.DataFrame(
    {
        "lemm": ["apples", "pears", "tasked", "children"],
        "word": ["apple", "pear", "task", "child"],
    }
)
print(lemm)
# Output
       lemm   word
0    apples  apple
1     pears   pear
2    tasked   task
3  children  child

df = pd.DataFrame(["apples", "children", "ipse", "lorem"])

print(df)
# Output
          0
0    apples
1  children
2      ipse
3     lorem

Make a dict (lemm:word) from lemm and replace with match in df:

df = df.applymap(lambda x: lemm.set_index("lemm").to_dict()["word"].get(x, pd.NA))

print(df)
# Output
       0
0  apple
1  child
2   <NA>
3   <NA>
user_112460560
Related