Want to use regular expression in Python to group words the way character classes and repetition quantifiers group characters

user_1945982 0
Viewed 159

I have a problem I am trying to solve in python, using re. I can use regex if I need the extended features, but my needs seem to be simple.

I have the following text for example:

one
one two 
one two three
one two three four
one two three four five

I want to get back the lines with 1, 2, or 3 words on them, bypassing the rest, so in the above example text, the first 3 lines will match, and the others will not.

The text is generated by the tesseract ocr utility, so the text can be just about anything, garbled mistakes and all, so I am finding words using \S in a character class like this: [\S]+

I have the following regular expression that works:

^[\S]+[ ]?[\S]+?[ ]?[\S]+?[ ]?$

The problem is, I am in the prototype phase of development, and I'm guessing I will need to expand this to accept up to six words, and at the same time, skip lines with only one word.

The regex could easily be expanded to do this, but I am almost certain I will end up needing a regex to capture up to three lines in a row that meet my single-line criteria.

So, thanks to regular-expressions.info, I am learning more about regular expressions, and see how character classes -- [] and repetition quantifiers -- {} work.

What I want is a way to use character classes and repetition quantifiers to group words instead of characters.

I know I could to this all line-by-line in python, but using a regular expression will allow me to expand my solution as I encounter more ocr output in the tool I am building.

Any help on how to proceed?

---- Adding test code and pythex.org screenshot detailing differing behavior.

Wiktor's regex is ^\S+(?: \S+){0,2}$

Test code:

import re


def testre(pattern, text):
    p = re.compile(pattern, re.M)
    results = p.findall(text)
    print(f'Test Results: {results}')


txt = 'one\none two\none two three\none two three four\none two three four five\n'
pattern1 = r'^\S+(?: \S+){0,2}$'
print(f'Test string...\n{txt}')
print(f'Test regex: {pattern1}')
testre(pattern1, txt)

Runtime shows that Wiktor's expression works as expected:

Test string...
one
one two
one two three
one two three four
one two three four five

Test regex: ^\S+(?: \S+){0,2}$
Test Results: ['one', 'one two', 'one two three']

However, running this on pythex.org shows 'one two' not matching: enter image description here

Wiktor's regex works, but the different result seen on pythex.org is a little concerning. I hope to use pythex for testing purposes.

text-processing python regex
1 Answers

There are couple of things you should know about the ^[\S]+[ ]?[\S]+?[ ]?[\S]+?[ ]?$ pattern: 1) it won't match 1- or 2-char "words" becuse \S+, [\S]+? and [\S]+? here each require at least one non-whitespace chars, 2) you should not (over|ab)use character classes, [\S] = \S, but [\b] != \b and [.] != .. Only use character classes when necessary to avoid messing up the pattern.

You can use

^\S+(?: \S+){0,2}$          # One to three "word" string with a regular space between the words only
^\S+(?:\s\S+){0,2}$         # One to three "word" string with any whitespace between the words only
^\S+(?:\s+\S+){0,2}$        # One to three "word" string with one or more whitespace chars between the words only
^\s*\S+(?:\s+\S+){0,2}\s*$  # One to three "word" string with one or more whitespace chars between
                            #  the words and allowing leading/trailing whitespace in the string

NOTE: \S+ matches any 1+ non-whitespace chars, and it can match abc, abc123,, ---++===, etc. If you mean to match words (letters, digits, _), you may use \w+. If you want to only match letter words, use [^\W\d_]+.

Details

  • ^ - start of string
  • \s* - zero or more any whitespace chars
  • \S+ - one or more non-whitespace chars
  • (?:\s+\S+){0,2} - zero to two occurrences of one or more any whitespace chars and then one or more non-whitespace chars
  • \s* - zero or more any whitespace chars
  • $ - end of string.
user_38329700
Related