Splitting a string into consonant-vowel sequences

user_6293961 0
Viewed 804

I am trying to split a string as follows:

  1. Zero or more consonants followed by zero or more vowels are a taken as a token.
  2. All other characters are taken as a token.

For example, 'yes, oat is good' is split as ['ye', 's', ',', ' ', 'oa', 't', ' ', 'i', 's', ' ', 'goo', 'd'].

Trying regex re.compile(r'[bcdefghjklmnpqrstuvwxyz]*[aeiou]*').findall('yes, oat is good') gives me ['yes', '', '', 'oa', 't', '', 'i', 's', '', 'goo', 'd', '']. Why is 'yes' not split into 'ye' and 's'?

Then, trying re.compile(r'[bcdefghjklmnpqrstuvwxyz]*[aeiou]*|.').findall('yes, oat is good') gives me the same result. Why does it not capture ',' and ' '?

Finally, is there a way to avoid getting empty strings?

tokenize python-3.x split regex
1 Answers

You should not include the letter e as one of the consonants. Aside from that, you should use an alternation pattern to match all the other characters as a token. Also use a positive lookahead pattern to ensure the pattern that matches zero or more consonants followed by zero or more vowels matches at least one alphabet:

re.findall(r'[^a-z]|(?=[a-z])[bcdfghjklmnpqrstvwxyz]*[aeiou]*', 'yes, oat is good', re.I)

This returns:

['ye', 's', ',', ' ', 'oa', 't', ' ', 'i', 's', ' ', 'goo', 'd']
user_68909120
Related