Extract everything but a regex match

Viewed 48

I have a dataframe where each row contains an e-mail's raw text. I need to clean up the data to extract the following columns: From, To, CC, Subject, and the body of the text. The e-mails typically look like this:

From   : Vincent Adultman
To     : Business Person, 
Cc     : 
Subject: On the subject of business Transactions

Dear blabla,

We would like to bla bla to improve our bla bla by X%.


Thanks in advance

I was able to extract the first four columns using the following regex expression:

import pandas as pd
df = pd.DataFrame(data=data,columns=['text'],dtype='string')

df['from'] = df.loc[:,'text'].str.extract(pat=r'(\bFrom .+)')
df['to'] = df.loc[:,'text'].str.extract(pat=r'(\bTo .+)')
df['cc'] = df.loc[:,'text'].str.extract(pat=r'(\bCc .+)')
df['bcc'] = df.loc[:,'text'].str.extract(pat=r'(\bBcc .+)')
df['subject'] = df.loc[:,'text'].str.extract(pat=r'(\bSubject: .+)')

Now I am trying to extract the rest of the body that starts at Dear blabla. However since every e-mail is different, I can't go matching on Dear blabla.

How can I match all the text except the first four matches I have already done?

Here is what I have tried:

df.loc[:,'text'].str.extract(pat=r'^(\bFrom .+|\bTo .+|\bCc .+|Bcc .+|\bSubject .+)')
df.loc[:,'text'].str.extract(pat=r'^[(\bFrom .+|\bTo .+|\bCc .+|Bcc .+|\bSubject .+)]')

What am I doing wrong?

1 Answers

You can use

df['body'] = df['text'].str.replace(r'^(?:\n?(?:From|To|Cc|Subject)\s*:.*)+\s*', '')

See the regex demo.

Details

  • ^ - start of string
  • (?:\n?(?:From|To|Cc|Subject)\s*:.*)+ - one or more repetitions of
    • \n? - an optional newline, line feed char
    • (?:From|To|Cc|Subject) - either From, or To, Cc, Subject
    • \s*: - 0 or more whitespace chars and a : char
    • .* - any 0 or more chars other than line break chars, as many as possible
  • \s* - 0 or more whitespace chars.
Related