Regex Python pulling between ( )

user_8719373 0
Viewed 55

I am trying to find proper names and dates between parentheses in Python using regex, but for some reason nothing seems to be working for me. While I expect it is an easy answer, it is hanging me up. Can anyone offer a solution?

I am using Python 3.7. The dataset consists of the text extracted from scholarly articles. I want to extract author's name and date of publication that always comes at the end of cited sentences. I want to make this into a list that is then stored in a pandas dataframe.

A typical sentence would be,

"Correlational factors in youth violence are still rudimentarily understood and often default to SES variables (Sheffield, 1998)."

What I am trying to extract is the (Sheffield, 1998) type of data. It will always appear in that pattern. My expected output would be

[(Sheffield, 1998)]

My current code is:

import re

test_text = ['Sentence 1 (Author, 2019).', 
        'Sentence 2 (Another Author, 2020)',
        'Sentence 3 (First Author & Second Author, 2018)',
        'Sentence 4 (Author, 2019; Another Author, 2020; Fourth Author, 2017)']

test_list = []

for elem in test_text:
    test_run = re.findall(r'\((\D+), (\d+)\w*\)', str(elem))

    if test_run: #if something was found
       test_list.append(test_run) 

    print(elem) #print out to see what is going on

    print(str(test_run), '\n') #print out to see what is going on

print("FULL LIST OF PULL:\n", test_list)

Now when I run this, regex is pulling the first three sentence examples fine, but not the fourth sentence example.

I appreciate any advice or help. Still learning regex and python (as my code probably shows). If there is a better way to do this that does not involve regex, I am open to learning.

Thank you in advance.

python-3.x regex
3 Answers

One way is to use a pattern and extract what is in between parenthesis using a capturing group ([^()]+) and a negated character class matching any char except ( or ). This is a bit of a broad pattern:

\(([^()]+)\)

Regex demo

You could make it more specific by specifying what you would allow using a character class and repeating groups to get a bit closer to the format of the data:

\((\w+(?: [\w&]+)*, \d{4}(?:; \w+(?: [\w&]+)*, \d{4})*)\)
  • \( Match (
  • ( Capture group 1
    • \w+ Match 1+ word chars,
    • (?: [\w&]+)*, \d{4} repeat 0+ times a space, 1+ word chars or &, a space and 4 digits
    • (?:; \w+(?: [\w&]+)*, \d{4})* Repeat the previous pattern 0+ times preceded by ;
  • ) Close group
  • \) Match )

Regex demo

import re

test_text = ['Sentence 1 (Author, 2019).',
             'Sentence 2 (Another Author, 2020)',
             'Sentence 3 (First Author & Second Author, 2018)',
             'Sentence 4 (Author, 2019; Another Author, 2020; Fourth Author, 2017)']

test_list = []
pattern = r'\(([^()]+)\)'

for elem in test_text:
    for splitOne in re.search(pattern, elem).group(1).split(";"):
        for splitTwo in splitOne.split(":"):
            test_list.append(splitTwo.strip())

print("FULL LIST OF PULL:\n", test_list)

Output

FULL LIST OF PULL:
 ['Author, 2019', 'Another Author, 2020', 'First Author & Second Author, 2018', 'Author, 2019', 'Another Author, 2020', 'Fourth Author, 2017']

Or create a list of lists using for example this Python demo.

Another option is to make use of the PyPi regex module which supports using a quantifier in the lookbehind.

For example:

(?<=\([^()]*)\w+(?: [\w&]+)*, \d+(?=[^\r\n()]*\))

The pattern asserts an opening ( on the left and asserts a closing ) on the right where the pattern matching the word chars and the digits are matched in the middle of the pattern.

Regex demo (with Javascript selected for demo purpose only) | Or see a Python demo

user_54249880

You could use the regular expression

\(([^()]+), *(\d{4})\)\.? *$

Demo.

The regex can be made self-documenting by writing it in free-spacing mode:

/
\(        # match '('
([^()]+)  # match 1+ chars other than '(' and ')' in capture group 1
,\ *      # match ',' followed by 0+ spaces
(\d{4})   # match four digits in capture group 2
\)        # match ')'      
\.?       # optionally match '.'
\ *       # match 0+ spaces
$         # match end-of-line
/x        # free-spacing regex definition mode

Note that in free-spacing mode spaces are removed before the expression is parsed. Spaces intended to be present must therefore be protected from removal. I've chosen to escape them but they could instead be put in a character class [ ] or replaced by [[:space:]], \p{Space} or, if appropriate, \s.

user_2569700

It's failing because your regex includes the parentheses, so it really only works for one (name, year) pair per parenthesized expression.

user_862330
Related