Multiline regex match retrieving line numbers and matches

user_5423415 0
Viewed 561

I'm attempting to iterate over all lines in a file to match a pattern that could;

  1. Occur anywhere in the file
  2. Occur multiple times in the same file
  3. Occur multiple times on the same line
  4. The string I'm searching for could be spread across multiple lines for one regex pattern

An example input would be;

new File()
new
File()
there is a new File()
new
    
    
    
File()
there is not a matching pattern here File() new
new File() test new File() occurs twice on this line

Example output would be;

new File() Found on line 1  
new File() Found on lines 2 & 3 
new File() Found on line 4 
new File() Found on lines 5 & 9 
new File() Found on line 11
new File() Found on line 11 
6 occurrences of new File() pattern in test.txt (Filename)

The regex pattern would look something like;

pattern = r'new\s+File\s*\({1}\s*\){1}'

Looking at the docs here, I can see match, findall and finditer all return matches at the beginning of strings but I don't see a way of using the search function which looks at any location for a regex where the string we're searching for is over multiple lines (Number four in my requirements above).

Simple enough to match more than one occurence of the regex per line with;

example input:

line = "new File() new File()"

Code:

i = 0
matches = []
while i < len(line):
    while line:
        matchObj = re.search(r"new\s+File\s*\({1}\s*\){1}", line, re.MULTILINE | re.DOTALL)
        if matchObj:
            line = line[matchObj.end():]
            matches.append(matchObj.group())

print(matches)

Prints the following matches - Not including line numbers ect for now:

['new File()', 'new File()']

Is there a way to do what I'm looking for with Python's regex?

python-3.x python regex
2 Answers

you could first find all \n characters in the text and their respective position/character index. since each \n...well...starts a new line, the index of each value in this list indicates the line number the found \n character terminates. then search all occurrences of you pattern and use the aforementioned list to look up the start/end position of the match...

import re
import bisect

text = """new 
File()
aa new File()
new
File()
there is a new File() and new
File() again
new
    
    
    
File()
there is not a matching pattern here File() new
new File() test new File() occurs twice on this line
"""

# character indices of all \n characters in text
nl = [m.start() for m in re.finditer("\n", text, re.MULTILINE|re.DOTALL)]

matches = list(re.finditer(r"(new\s+File\(\))", text, re.MULTILINE|re.DOTALL))
match_count = 0
for m in matches:
    match_count += 1
    r = range(bisect.bisect(nl, m.start()-1), bisect.bisect(nl, m.end()-1)+1)
    print(re.sub(r"\s+", " ", m.group(1), re.DOTALL), "found on line(s)", *r)
print(f"{match_count} occurrences of new File() found in file....")

output:

new File() found on line(s) 0 1
new File() found on line(s) 2
new File() found on line(s) 3 4
new File() found on line(s) 5
new File() found on line(s) 5 6
new File() found on line(s) 7 8 9 10 11
new File() found on line(s) 13
new File() found on line(s) 13
8 occurrences of new File() found in file....
user_126937280

You can count the number of newlines before the match, and then count the number of newlines in the match value, and combine the line numbers: See the Python demo:

import re
s='new File()\nnew\nFile()\nthere is a new File()\nnew\n \n \n \nFile()\nthere is not a matching pattern here File() new\nnew File() test new File() occurs twice on this line'
pattern = r'new\s+File\s*\(\s*\)'
for m in re.finditer(pattern, s):
    linenums = [s[:m.start()].count('\n') + 1]
    for _ in range(m.group().count('\n')):
        linenums.append(linenums[-1] + 1)
    print('{} Found on line {}'.format(re.sub(r'\s+', ' ', m.group()), ", ".join(map(str,linenums))))

See the online Python demo.

Output:

new File() Found on line 1
new File() Found on line 2, 3
new File() Found on line 4
new File() Found on line 5, 6, 7, 8, 9
new File() Found on line 11
new File() Found on line 11
user_38329700
Related