how to write a regular expression to match a small part of a repeating pattern?

user_11970777 0
Viewed 75

I have the following pattern to match :

(10,'more random stuff 21325','random stuff','2014-10-26 04:50:23','','uca-default-u-kn','page')

For some context, it's part of a larger file , which contains many similar patterns separated by commas :

(10,'more random stuff 21325','random stuff','2014-10-26 04:50:23','','uca-default-u-kn','page'),
(11,'more random stuff 1nyny5','random stuff','2014-10-26 04:50:23','','uca-default-u-kn','subcat'),
(14,'more random stuff 21dd5','random stuff','2014-10-26 04:50:23','','uca-default-u-kn','page')

my goal is to ditch all patterns ending with 'page' and to keep the rest. For that, I'm trying to use regular expressions to identify those patterns. Here is the one I come out with for now :

"\(.*?,\'page\'\)"

However, it's not working as expected. In the following python code, I use this regex, and replace every match with an empty string :

import re

txt = "(10,'Redirects_from_moves','*..2NN:,@2.FBHRP:D6ܽ�','2014-10-26 04:50:23','','uca-default-u-kn','page'),"
txt += "(11,'Redirects_with_old_history','*..2NN:,@2.FBHRP:D6ܽ�','2010-08-26 22:38:36','','uca-default-u-kn','page'),"
txt += "(12,'Unprintworthy_redirects','*..2NN:,@2.FBHRP:D6ܽ�','2010-08-26 22:38:36','','uca-default-u-kn','subcat'),"
txt += "(13,'Anarchism','random_stuff','2020-01-23 13:27:44',' ','uca-default-u-kn','page'),"
txt += "(14,'Anti-capitalism','random_stuff','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),"
txt += "(15,'Anti-fascism','*D*L.8:NB\r�','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),"
txt += "(16,'Articles_containing_French-language_text','*D*L.8:NB\r�','2020-01-23 13:27:44','','uca-default-u-kn','page'),"
txt += "(17,'Articles_containing_French-language_text','*D*L.8:NB\r�','2020-01-23 13:27:44','','uca-default-u-kn','page')"

new_txt = re.sub("\(.*?,\'page\'\)", "",txt)

I was expecting that new_text would contains all patterns ending with 'subcat', and remove all patterns ending with 'page', however, I obtain :

new_txt = ,,,,

What's happening here ? How can I change my regex to obtain the desired result ?

python regex
2 Answers

We might be tempted to do a regex replacement here, but that would basically always leave open edge cases, as @Wiktor has correctly pointed out in a comment below. Instead, a more foolproof approach is to use re.findall and simply extract every tuple with does not end in 'page'. Here is an example:

parts = re.findall(r"\(\d+,'[^']*?'(?:,'[^']*?'){4},'(?!page')[^']*?'\),?", txt)
print(''.join(parts))

This prints:

(12,'Unprintworthy_redirects','*..2NN:,@2.FBHRP:D6ܽ�','2010-08-26 22:38:36','','uca-default-u-kn','subcat'),(14,'Anti-capitalism','random_stuff','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),(15,'Anti-fascism','DL.8:NB�','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),

The regex pattern used above just matches a leading number, followed by 5 singly quoted terms, and then a sixth singly quoted term which is not 'page'. Then, we string join the tuples in the list output to form a string.

user_18632290

What happens is that you concatenate the string, then then remove all until the first occurrence of ,'page') leaving only the trailing comma's.

Another workaround might be using a list of the strings, and join them with a newline instead of concatenating them.

Then use your pattern matching an optional comma and newline at the end to remove the line, leaving the ones that end with subcat

import re
lines = [
"(10,'Redirects_from_moves','*..2NN:,@2.FBHRP:D6ܽ�','2014-10-26 04:50:23','','uca-default-u-kn','page'),",
"(11,'Redirects_with_old_history','*..2NN:,@2.FBHRP:D6ܽ�','2010-08-26 22:38:36','','uca-default-u-kn','page'),",
"(12,'Unprintworthy_redirects','*..2NN:,@2.FBHRP:D6ܽ�','2010-08-26 22:38:36','','uca-default-u-kn','subcat'),",
"(13,'Anarchism','random_stuff','2020-01-23 13:27:44',' ','uca-default-u-kn','page'),",
"(14,'Anti-capitalism','random_stuff','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),",
"(15,'Anti-fascism','*D*L.8:NB\r�','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),",
"(16,'Articles_containing_French-language_text','*D*L.8:NB\r�','2020-01-23 13:27:44','','uca-default-u-kn','page'),",
"(17,'Articles_containing_French-language_text','*D*L.8:NB\r�','2020-01-23 13:27:44','','uca-default-u-kn','page')"
]

new_txt = re.sub("\(.*,'page'\)(?:,\n)?", "", '\n'.join(lines))
print(new_txt)

Output

(12,'Unprintworthy_redirects','*..2NN:,@2.FBHRP:D6ܽ�','2010-08-26 22:38:36','','uca-default-u-kn','subcat'),
(14,'Anti-capitalism','random_stuff','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),
�','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),

Or you can use a list comprehension to keep the lines that do not match the pattern.

result = [line for line in lines if not re.match(r"\(.*,'page'\),?$", line)]
print('\n'.join(result))

Output

(12,'Unprintworthy_redirects','*..2NN:,@2.FBHRP:D6ܽ�','2010-08-26 22:38:36','','uca-default-u-kn','subcat'),
(14,'Anti-capitalism','random_stuff','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),
�','2020-01-23 13:27:44','','uca-default-u-kn','subcat'),

Another option to match the parts that end with 'page') for the example data:

\(\d+,[^)]*(?:\)(?!,\s*\(\d+,)[^)]*)*,'page'\),?

The pattern matches:

  • \(\d+, Match ( followed by 1+ digits and a comma
  • [^)]* Optionally match any char except )
  • (?: Non capture group
    • \)(?!,\s*\(\d+,)[^)]* Only match a ) when not directly followed by the pattern ,\s*\(\d+, which matches the start of the parts in the example data
  • )* Close group and optionally repeat
  • ,'page'\),? Match ,'page') with an optional comma

Regex demo

user_54249880
Related