Pandas regex to extract everything after and before two different symbols

user_10829044 0
Viewed 854

I have a dataframe like as shown below

df = pd.DataFrame({'person_id': [11,11,11],'text':['DOSE: 667 mg - TDS with food - Inject  hypo > 4 test value here','DOSE: 667 mg - TDS with food - Tube','DOSE: 667 mg - TDS with food - PO'],'Summary':['Test','Test1','Test2']})

From the above data frame, I would like to extract everything that

a) comes after DOSE: and before 1st hyphen -

b) Again extract everything that comes after 2nd hyphen -.

I was trying something like below

df['text'].str.extract('(\d+[A-Za-ZS]*(\-))',expand=True)  # doesn't work 
s1 = df['text'].str.split() # I tried using `str.split`
s1[0][1]  # it goes on like for loop which is not elegant.

Can help me with the above?

I expect my output to be like as shown below

enter image description here

pandas regex-lookarounds dataframe python-3.x regex
4 Answers

You can use

import pandas as pd
df = pd.DataFrame({'person_id': [11,11,11],'text':['DOSE: 667 mg - TDS with food - Inject','DOSE: 667 mg - TDS with food - Tube','DOSE: 667 mg - TDS with food - PO'],'Summary':['Test','Test1','Test2']})
df['text'].str.replace(r'^DOSE:\s*([^-]*)-[^-]*-\s*(.*)', r'\1\2')
# 0    667 mg Inject
# 1      667 mg Tube
# 2        667 mg PO
# Name: text, dtype: object

The regex is

^DOSE:\s*([^-]*)-[^-]*-\s*(.*)

See the regex demo. Details:

  • ^DOSE: - DOSE: at the start of string
  • \s* - 0+ whitespaces
  • ([^-]*) - Group 1 (\1 refers to this group value from the replacement pattern): any 0 or more chars other than a -
  • - - a hyphen
  • [^-]* - 0+ chars other than -
  • - - a -
  • \s* - 0+ whitespaces
  • (.*) - Capturing group 2 (\2 refers to this group value from the replacement pattern): any zero or more chars other than line break chars, as many as possible.
user_38329700

Wiktor has shown how to pick and chose substring in text column using str.replace.

Here is how to get your expected output:

>>> df = pd.DataFrame({'person_id': [11,11,11],'text':['DOSE: 667 mg - TDS with food - Inject','DOSE: 667 mg - TDS with food - Tube','DOSE: 667 mg - TDS with food - PO'],'Summary':['Test','Test1','Test2']})
>>> df['text'] = df['text'].str.replace(r'\bDOSE:\s*([^-]*)-[^-]*-\s*(.*)', r'\1\2')
>>> print( df.filter(['person_id', 'text', 'Summary']) )

   person_id           text Summary
0         11  667 mg Inject    Test
1         11    667 mg Tube   Test1
2         11      667 mg PO   Test2

Code Demo

user_5482250

You probably don't need regex here. You can use str.split() and join 0th and 2nd item.

df['text_new'] = df.text.apply(lambda x: " ".join(map(x.replace("DOSE: ", "").split("-").__getitem__, [0, 2])))
user_13690070

Lets use str.replace

df.text.str.replace(r'DOSE:\s|\s-\sTDS\swith\sfood\s-','')
user_89869750
Related