Spyder breaks while reading large text file

Viewed 22

I am trying to extract specific lines from a huge text file (over 1.3 GB) with millions of lines. I have tried something like this:

import re
pattern='Blah blah'
my_list=[]
with open(r'C:\Users\Desktop\log.txt') as my_file:
    for lines in my_file:
        k=re.search(pattern, lines)
        if k!=None:
            my_list.append(lines)
print(my_list)

The problem is that, spyder breaks and exits before completing the loop. Can you suggest a more memory efficient way to handle this operation?

0 Answers
Related