match part of text based on another column

user_10718214 0
Viewed 149

I have this dataset;

text                                    num

test one 3.5 and 60 test tow            3.5/60
test one 3/4 test tow                     3/4
test one 5.0 test 10 tow                  5.0

I need to remove the number from text column if it match in num so I have did this:

df['text']=[re.sub('{}'.format(number), '', the_text) for the_text, number in zip(df['text'], df['num'])]

the result become like

text                                    num

test one 3.5 and 60 test tow            3.5/60
test one test tow                         3/4
test one test 10 tow                      5.0

as you can see the matched number was removed except the first row because it is not an exact match. so I want to use contains function match or anything similar to match part of a text.

I expect the first row will become

test one and test tow

I have did this but get error:

[re.sub(r"\b{}\b".format(word), "", the_text) for the_text, word in zip(df['text'], word='/'.join([r'{}'.format(words) for words in df['num']]) )]

TypeError: zip() takes no keyword arguments

any help?

pandas match python regex
3 Answers

You may use

df['text'] = df.apply(lambda x: re.sub(r'(?<!\d)(?<!\d\.)(?:{}|{})(?!\.?\d)'.format(re.escape(x['num']), '|'.join([re.escape(l) for l in x['num'].split('/')])), '', x['text']), axis=1)

Thanks to df.apply and axis=1, we iterate over all rows.

The regex is made dynamically based on the value in the num column and is applied to text column.

r'(?<!\d)(?<!\d\.)(?:{}|{})(?!\.?\d)'.format(re.escape(x['num']), '|'.join([re.escape(l) for l in x['num'].split('/')])) creates a regex like

(?<!\d)(?<!\d\.)(?:3/4|3|4)(?!\.?\d)

that matches both full value in the num column and the numbers in between / separately.

The (?<!\d)(?<!\d\.) are a lookbehind sequence that fail the match if there is a digit or a digit + dot immediately to the left of the current position, and (?!\.?\d) fails the match if there is a digit or a dot + digit immediately to the right of the current position effectively disallowing number matches in longer numbers.

user_38329700

This works:

import re

txt='''\
text                                    num

test one 3.5 and 60 test tow            3.5/60
test one 3/4 test tow                     3/4
test one 5.0 test tow                     5.0'''

for line in txt.splitlines():
    m=re.search(r'^(.*?[ \t]{2,}(?=\d))([0-9.\/]+)$', line)
    if m:
        a,_,b=m.group(2).partition('/')
        if re.search(fr'\b{m.group(2)}\b', m.group(1)):
            l=len(m.group(1))
            s=re.sub(fr'[ ]?\b{m.group(2)}\b', '', m.group(1))
            line=s+' '*(l-len(s))+m.group(2)
        elif re.search(fr'{a}[^/]+{b}', m.group(1)):
            l=len(m.group(1))
            s=re.sub(fr'[ ]?\b{a}\b','',m.group(1))
            s=re.sub(fr'[ ]?\b{b}\b','',s)
            line=s+' '*(l-len(s))+m.group(2)
                
    print(line)

Prints:

text                                    num

test one and test tow                   3.5/60
test one test tow                         3/4
test one test tow                         5.0
user_2986070

Create a list of the numbers and also add the /

nums = '|'.join(df['num'].tolist()).replace('/', '|') + '|/'
nums
'3.5|60|3|4|5.0|/'

Then str replace

df['text'].str.replace(nums, '')

0    test one  and  test tow
1         test one  test tow
2         test one  test tow
user_60319950
Related