Split string on multiple delimiters but not on white space alone

user_1711146 0
Viewed 125

I'm doing

delimiters = [r'\r\n', r'\.\.\.', r'\W']
pattern = regex.compile(r'(' + r'|'.join(delimiters) + r')', flags=regex.V1)
pattern.split(s)

to split s on multiple delimiters (using parentheses to retain delimiters in the output).

How can I prevent splitting on white space when it occurs alone? For example,

'abc, def , geh,, , ijk      lmn \n opq'

should give

`['abc', ', ', 'def', ' , ', 'geh', ',, , ', 'ijk      lmn', ' \n ', 'opq']

That is, when white space occurs together with another delimiter, the split should be performed but not when it there is only white space between the tokens. By white space I really only mean white space ' ', not any other tokens such as line break.s

(These are actually two questions, as right now I'm getting ['abc', ',', '', ' ', 'def', ' ', 'geh'], that is, multiple delimiters come out separately, while I would like them to come out aggregated.)

python regex
2 Answers

With regex module you could use:

import regex
arr = ['abc,, def', 'abc, def geh', 'abc  def', 'abc, def , geh,, , ijk      lmn \n opq']
res = [regex.split(r'\b(?=\W)(?! +\b)|(?<=\b\W*[^\w ]+\W*\b)',x) for x in arr]
print(res)

Prints:

[['abc', ',, ', 'def'], ['abc', ', ', 'def geh'], ['abc  def'], ['abc', ', ', 'def', ' , ', 'geh', ',, , ', 'ijk      lmn', ' \n ', 'opq']]

The pattern matches:

  • \b - Word boundary.
  • (?=\W) - A positive lookahead of a non-word character.
  • (?!\s\b) - Negative lookahead for a space character and a word boundary.
  • | - Or
  • (?<=\b\W*[^\w\s]+\W*\b) - A positive lookbehind for a word-boundary, zero or more non-word characters, at least one character other than wordcharacter or whitespace character and followed by (greedy) non-word characters if possible and a word-boundary.
user_97581940

You may use

import re
delimiters = [r'\r\n', r'\.\.\.', r'[^\w\s]']
d = r"|".join(delimiters)
pattern = re.compile(rf'((?:\s*(?:{d}))+\s*)')
s = 'abc, def , geh,, , ijk      lmn'
print( pattern.split(s) )
# => ['abc', ', ', 'def', ' , ', 'geh', ',, , ', 'ijk      lmn']

See the Python demo.

See the regex demo meaning

  • (?:\s*(?:\r\n|\.\.\.|[^\w\s]))+ - one or more repetitions of:
    • \s* - 0+ whitespaces
    • (?:\r\n|\.\.\.|[^\w\s]) - one of your delimiter patterns
  • \s* - 0+ whitespaces

Note \W is replaced with [^\w\s] to match any punctuation (any non-word char but whitespace chars).

user_38329700
Related