A way to match Lisp characters containing a parenthesis

user_11723707 0
Viewed 95

I am writing a Lisp parser in Python as part of a compiler project. My tokenizer works something like this:

def tokenize(filename):
    with open(filename, "r") as file:
        comment_free = remove_comments(file.read())
    return comment_free.replace("(", " ( ").replace(")", " ) ").split()

Lisp characters are a little bit different: the character a would be #\a, ( would be #\(, et cetera. For characters that contain a left or right parenthesis, they are wrongly split apart by the tokenizer into two separate pieces (#\ and ( or )).

I can match the character indicator with [^#\] and parenthesis with \(|\), but how would I create a pattern that checks if a part of the code does not start with a character indicator and follows with a left or right parenthesis? If I can make such a pattern I can replace all such instances with a parenthesis that has spaces around it. I thought that ([^#\\])(\(|\))* would work but it does not work in all cases. Does anyone proficient with regex know how to figure this out?

regex-lookarounds match parsing python regex
1 Answers

how would I create a pattern that checks if a part of the code does not start with a character indicator and follows with a left or right parenthesis?

You might use a negative lookbehind to check that the indicator is not on the left, and match either ( or ) using a character class instead of an alternation.

(?<!#\\)[()]

Explanation

  • (?<!#\\) Negative lookbehind, assert what is directly to the left is not #\
  • [()] Match either ( or )

Regex demo

If the indicator is either # or \\ you could use 2 character classes:

(?<![#\\])[()]

Regex demo

user_54249880
Related