how to remove exact match of words in data frame in python?

user_13990072 0
Viewed 770

Suppose the following data frame with one column called < game >:

df:
   game
0  juegos blue
1  juego red
2  juegos yellow

I would like to remove the words from the following list of stop words:

stopWords = ['juego','juegos']

and the desired result is:

df:
   game
0  blue
1  red
2  yellow

I tried that:

df['game'] = df['game'].str.replace("|".join(stopWords ), " ")

The function works but it removes 'juego' from the entry "juegos" leaving an 's' alone:

df:
   game
0  s blue
1   red
2  s yellow

Is there a way to remove words only if there is an exact match?

python regex string
2 Answers

You can just do it with pandas DataFrame.replace()

In [1]: import pandas as pd 
   ...: df = pd.DataFrame({'game': ['juegos blue', 'juego red', 'juegos yellow']}) 
   ...: stop_words = [r'juego\b', r'juegos\b'] 
   ...: df.replace(to_replace={'game': '|'.join(stop_words)}, value='', regex=True, inplace=True) 
   ...: df                                                                                                                                                    
Out[1]: 
      game
0     blue
1      red
2   yellow

In [2]: df = pd.DataFrame({'game': ['juegos blue', 'juego red', 'juegos yellow']}) 
   ...: stop_words = [r'juego\b'] 
   ...: df.replace(to_replace={'game': '|'.join(stop_words)}, value='', regex=True, inplace=True) 
   ...: df                                                                                                                                                    
Out[2]: 
            game
0    juegos blue
1            red
2  juegos yellow

Assuming that stop 'words' end with the word boundary \b.

user_114914300

Python string replace won't work, but the regex module will. You're going to need to add some markup to the string to make the regex look for complete words. For instance, maybe you know it's a complete word because it's followed by a period ., comma ,, or whitespace of any type \s, or an endline $. \b is the regex pattern for word boundaries.

import re
s1 = df['game'].str
for sw in stopWords:
    s1 = re.sub(r'{0}\b'.format(sw), '', s1)
df['game'].str = s1

(I stole the \b from this other good answer.)

Keeping old code in case you're interested. This one also removes whitespace, comma, or period directly after the matching word, which is not what you asked for but might be useful.

import re
s1 = df['game'].str
for sw in stopWords:
    s1 = re.sub(r'{0}([.,\s]|$)'.format(sw), '', s1)
df['game'].str = s1
user_7563290
Related