How to optimize the python code written using regex and for loops?

user_11280350 0
Viewed 93

I have two lists and I need to perform a string match. I have used three for loops and re.pattern to solve. I am getting the expected using existing code (part1), but I need to optimized the code (part2) as it takes a longer time when I apply for lengthy data.

part1

texts = ['foo abc', 'foobar xyz', 'xyz baz32', 'baz 45','fooz','bazzar','foo baz']
terms = ['foo','baz','apple']

output_list = []
for term in terms:
    pattern_term = r'\b(?:{})\b'.format(term)
    try:
        for i in range(len(texts)):
            line_text = texts[i]
            for match in re.finditer(pattern_term, line_text):
                start_index = match.start()
                output_list.append([i, start_index, line_text[start_index:], term])
    except:
        pass

output: Explaination fo columns names :

Index = index of texts when pattern matches

Start_index = start index where pattern matches inside text

Match_text = complete text of that matching

Match_term = term with it matches

pd.DataFrame(output_list, columns = ['Index', 'Start_index', 'Match_text', 'Match_term'])

Index   Start_index     Match_text  Match_term
0   0   0   foo abc     foo
1   6   0   foo baz     foo
2   3   0   baz 45      baz
3   6   4   baz         baz

I have tried the following code (part2), but its output is partial:

part 2

df = pd.DataFrame({'Match_text': texts})
pat = r'\b(?:{})\b'.format('|'.join(terms))
df[df['Match_text'].str.contains(pat)]

output

    Match_text
0   foo abc
3   baz 45
6   foo baz
pandas python-3.x optimization regex
1 Answers

Your code is already good since you need to find occurrences of whole words inside longer strings, and you create the regex pattern before the loop where the texts are processed with the regex.

The regex already is good, the only thing about it is the redundant non-capturing group that you may discard because you check term by term, there is no alternation inside the group. You might also compile the regex:

pattern_term = re.compile(r'\b{}\b'.format(term))

Then, you may get rid of temporary variables in the for loop:

for i in range(len(texts)):
    for match in pattern_term.finditer(texts[i]):
        output_list.append([i, match.start(), texts[i][match.start():], term])
user_38329700
Related