how do I remove the first instance of a regex capture group from strings in a pandas series?

user_875343 0
Viewed 114

I want to remove the first instance of a regex capture group from a series of strings in pandas, the inverse of pandas.Series.str.extract. This works fine with extract in the following code, where I extract the first word that begins with a double underscore:

import pandas as pd

regex = r"^(?:.*?)(__\S+)"

t = pd.Series(['no match', '__match at start', 'match __in_the middle', 'the end __matches', 'string __with two __captures'])
t.str.extract(regex)

           0
0        NaN
1    __match
2   __in_the
3  __matches
4     __with
dtype: object

but if I do pandas.Series.str.replace with an empty string then it replaces the noncapturing group as well:

t.str.replace(regex, '')

0        no match
1        at start
2          middle
3
4  two __captures
dtype: object

How do I get the following output?

0                no match
1                at start
2           match  middle
3                the end
4  string  two __captures
dtype: object
pandas python regex
2 Answers

You can use

t.str.replace(r'^(.*?)__\S+', r'\1', regex=True)

Or:

t.str.replace(r'__\S+', '', n=1, regex=True)

See the Series.str.replace documenation:

Replace each occurrence of pattern/regex in the Series/Index.
...
nint, default -1 (all)
     Number of replacements to make from start.

Output:

0         no match
1         at start
2    match  middle
3         the end 
dtype: object

Details:

  • ^ - start of string
  • (.*?) - Capturing group 1: any zero or more chars other than line break chars as few as possible
  • __ - two underscores
  • \S+ - one or more non-whitespace chars.

The (?:...) is a non-capturing group, and the text it matches is still consumed (i.e. added to the match value and is thus affected during the regex replace operation). "Non-capturing" does not mean "non-matching".

The \1 in the replacement pattern is the replacement backreference that refers to the text captured into capturing group 1.

The point is that we capture into a group and restore with a backreference used in the replacement pattern the text that we want to keep. The text we want to replace/remove is just matched, not captured.

user_38329700

According to pandas's docs you can specify the number of replacements you want to to make from start. If it works anything like Python's re.sub then you can use @Wiktor Stribiżew code with one modification

t.str.replace(r'__\S+', '', 1, regex=True)

If this not how things work in pandas then

for s in t:
    x = re.sub(r"__\S+", "", s, 1)
user_190048840
Related