python regex where a set of options can occur at most once in a list, in any order

Viewed 1390

I'm wondering if there's any way in python or perl to build a regex where you can define a set of options can appear at most once in any order. So for example I would like a derivative of foo(?: [abc])*, where a, b, c could only appear once. So:

foo a b c
foo b c a
foo a b
foo b

would all be valid, but

foo b b

would not be

8 Answers

You may use this regex with a capture group and a negative lookahead:

For Perl, you can use this variant with forward referencing:

^foo((?!.*\1) [abc])+$

RegEx Demo

RegEx Details:

  • ^: Start
  • foo: Match foo
  • (: Start a capture group #1
    • (?!.*\1): Negative lookahead to assert that we don't match what we have in capture group #1 anywhere in input
    • [abc]: Match a space followed by a or b or c
  • )+: End capture group #1. Repeat this group 1+ times
  • $: End

As mentioned earlier, this regex is using a feature called Forward Referencing which is a back-reference to a group that appears later in the regex pattern. JGsoft, .NET, Java, Perl, PCRE, PHP, Delphi, and Ruby allow forward references but Python doesn't.


Here is a work-around of same regex for Python that doesn't use forward referencing:

^foo(?!.* ([abc]).*\1)(?: [abc])+$

Here we use a negative lookahead before repeated group to check and fail the match if there is any repeat of allowed substrings i.e. [abc].

RegEx Demo 2

You can assert that there is no match for a second match for a space and a letter at the right:

foo(?!(?: [abc])*( [abc])(?: [abc])*\1)(?: [abc])*
  • foo Match literally
  • (?! Negative lookahead
    • (?: [abc])* Match optional repetitions of a space and a b or c
    • ( [abc]) Capture group, use to compare with a backreference for the same
    • (?: [abc])* Match again a space and either a b or c
    • \1 Backreference to group 1
  • ) Close lookahead
  • (?: [abc])* Match optional repetitions or a space and either a b or c

Regex demo

If you don't want to match only foo, you can change the quantifier to 1 or more (?: [abc])+


A variant in perl reusing the first subpattern using (?1) which refers to the capture group ([abc])

^foo ([abc])(?: (?!\1)((?1))(?: (?!\1|\2)(?1))?)?$

Regex demo

If it doesn't have to be a regex:

import collections

# python >=3.10
def is_a_match(sentence):
    words = sentence.split()
    return (
      (len(words) > 0)
      and (words[0] == 'foo')
      and (collections.Counter(words) <= collections.Counter(['foo', 'a', 'b', 'c']))
    )

# python <3.10
def is_a_match(sentence):
    words = sentence.split()
    return (
      (len(words) > 0)
      and (words[0] == 'foo')
      and not (collections.Counter(words) - collections.Counter(['foo', 'a', 'b', 'c']))
    )

# TESTING
#foo a b c True
#foo b c a True
#foo a b True
#foo b True
#foo b b False

Or with a set and the walrus operator:

def is_a_match(sentence):
    words = sentence.split()
    return (
      (len(words) > 0)
      and (words[0] == 'foo')
      and (
        (s := set(words[1:])) <= set(['a', 'b', 'c'])
        and len(s) == len(words) - 1
      )
    )

You can do it using references to previously captured groups.

foo(?: ([abc]))?(?: (?!\1)([abc]))?(?: (?!\1|\2)([abc]))?$

This gets quite long with many options. Such a regex can be generated dynamically, if necessary.

def match_sequence_without_repeats(options, seperator):
    def prevent_previous(n):
        if n == 0:
            return ""
        groups = "".join(rf"\{i}" for i in range(1, n + 1))
        return f"(?!{groups})"

    return "".join(
        f"(?:{seperator}{prevent_previous(i)}([{options}]))?"
        for i in range(len(options))
    )


print(f"foo{match_sequence_without_repeats('abc', ' ')}$")

Here is a modified version of anubhava's answer, using a backreference (which works in Python, and is easier to understand at least for me) instead of a forward reference.

Match using [abc] inside a capturing group, then check that the text matched by the capturing group does not appear again anywhere after it:

^foo(?:( [abc])(?!.*\1))+$

regex demo

  • ^: Start
  • foo: Match foo
  • (?:: Start non-capturing group (?:( [abc])(?!.*\1))
    • ( [abc]): Capturing Group 1, matching a space followed by either a, b, or c
    • (?!.*\1): Negative lookahead, failing to match if the text matched by the first capturing group occurs after zero or more characters matched by .
  • )+: End non-capturing group and match it 1 or more times
  • $: End

I have assumed that the elements of the string can be in any order and appear any number of times. For example, 'a foo' should match and 'a foo b foo' should not.

You can do that with a series of alternations employing lookaheads, one for each substring of interest, but it becomes a bit of a dog's breakfast when there are many strings to consider. Let's suppose you wanted to match zero or one "foo"'s and/or zero or one "a"'s. You could use the following regular expression:

^(?:(?!.*\bfoo\b)|(?=(?:(?!\bfoo\b).)*\bfoo\b(?!(.*\bfoo\b))))(?:(?!.*\ba\b)|(?=(?:(?!\ba\b).)*\ba\b(?!(.*\ba\b))))

Start your engine!

This matches, for example, 'foofoo', 'aa' and afooa. If they are not to be matched remove the word breaks (\b).

Notice that this expression begins by asserting the start of the string (^) followed by two positive lookaheads, one for 'foo' and one for 'a'. To also check for, say, 'c' one would tack on

(?:(?!.*\bc\b)|(?=(?:(?!\bc\b).)*\bc\b(?!(.*\bc\b))))

which is the same as

(?:(?!.*\ba\b)|(?=(?:(?!\ba\b).)*\ba\b(?!(.*\ba\b))))

with \ba\b changed to \bc\b.

It would be nice to be able to use back-references but I don't see how that could be done.

By hovering over the regular expression in the link an explanation is provided for each element of the expression. (If this is not clear I am referring to the cursor.)

Note that

(?!\bfoo\b).

matches a character provided it does not begin the word 'foo'. Therefore

(?:(?!\bfoo\b).)*

matches a substring that does not contain 'foo' and does not end with 'f' followed by 'oo'.

Would I advocate this approach in practice, as opposed to using simple string methods? Let me ponder that.

If the order of the strings doesn't matter, and you want to make sure every string occurs only once, you can turn the list into a set in Python:

my_lst = ['a', 'a', 'b', 'c']
my_set = set(lst)

print(my_set)
# {'a', 'c', 'b'}

There is not much to add to the above answers except here is a regex that does not use back or forward references. Instead it uses 3 separate negative lookahead assertions to ensure that the input does not contain 2 occurrences of either a or b or c. The regex also allows for liberal uses of spaces.

^foo(?![^a]*a[^a]*a)(?![^b]*b[^b]*b)(?![^c]*c[^c]*c)( +[abc])* *$

See Regex Demo

  1. ^ - Matches start of string
  2. (?![^a]*a[^a]*a) - Negative lookahead assertion that what follows does not contain two occurrences of a
  3. (?![^b]*b[^b]*b) - Negative lookahead assertion that what follows does not contain two occurrences of b
  4. (?![^c]*c[^c]*c) - Negative lookahead assertion that what follows does not contain two occurrences of c
  5. ( +[abc])* - Matches 0 or more occurrences of: 1 or more spaces followed by an a or b or c
  6. * - Matches 0 or more occurrences of space 7 $ - Matches the end of the string

The regex looks "clunky" but is very straightforward. With input foo a b c the successful match is done in 35 steps and with input foo b b the unsuccessful match is done in 13 steps. Thich compares favorably with the other answers.

Related