Finding the line of a specific string and reading the text file after that line

user_6935661 0
Viewed 130

I have a text file (~20MB) from which I want to extract some information. The info I am interested in looks like this:

   Generate :
 MESH :     Cartesian
   1.00000   0.00000   0.00000
   0.00000   0.84680   0.00000
   0.00000   0.00000   0.80724
 MESH : 4 unique points
               x           y           z        Weight
    1      0.000000    0.000000    0.000000     0.3906
    2      0.125000    0.000000    0.000000     0.7812
    3      0.250000    0.000000    0.000000     0.7812
    4      0.375000    0.000000    0.000000     0.7812

I want to save the x,y,z columns to an array after the second occurrence of the string 'MESH'. I tried using regex, but my solution saves the result as a list and makes it too complicated to call these values for future purposes. Here is my attempt:

import re

line_number = 0
mesh_list = []
Qp = []
with open('out.test','r') as f:
    for line in f:
        line_number +=1
        if 'MESH' in line:
            mesh_list.append([(line_number),line.rstrip()])

point_info = mesh_list[1]
output_line = point_info[0]             ## Line number where MESH appears the second time.
point_list = point_info[1].split()
num_of_points = int(point_list[1])      ## Get number of unique points.

with open('out.test','r') as f:
    for i, line in enumerate(f):
        if output_line+1 <= i <= output_line+num_of_points:
            Qp.append([line])

print(Qp)

At this point, 'Qp' has all the lines I need, but how can I separate x,y,z columns from this chunk? Would it be easier with pandas?

pandas python regex
2 Answers

You can use pd.read_csv with custom skiprows= and sep= parameters:

import re
import pandas as pd

r = re.compile(r"MESH : \d+ unique points")

line_counter = 0
with open("your_file.txt", "r") as f_in:
    for l in f_in:
        line_counter += 1
        if r.search(l):
            break

df = pd.read_csv("your_file.txt", skiprows=line_counter, sep=r"\s+")
print(df)

Prints:

       x    y    z  Weight
1  0.000  0.0  0.0  0.3906
2  0.125  0.0  0.0  0.7812
3  0.250  0.0  0.0  0.7812
4  0.375  0.0  0.0  0.7812
user_100359850

To get x, y, z (line number and Weight) from Qp, which is a list of one-element lists, as tuples (converting the tuples to lists is trivial) you can try:

>>> Qp
[['    1      0.000000    0.000000    0.000000     0.3906\n'], ['    2      0.125000    0.000000    0.000000     0.7812\n'], ['    3      0.250000    0.000000    0.000000     0.7812\n'], ['    4      0.375000    0.000000    0.000000     0.7812\n']]

>>> lno, x, y, z, Weight  = zip(*(line[0].split() for line in Qp))
>>> lno
('1', '2', '3', '4')
>>> x
('0.000000', '0.125000', '0.250000', '0.375000')
>>> y
('0.000000', '0.000000', '0.000000', '0.000000')
>>> z
('0.000000', '0.000000', '0.000000', '0.000000')
>>> Weight
('0.3906', '0.7812', '0.7812', '0.7812')

For floats instead of strs:

>>> lno, x, y, z, Weight = zip(*((float(a) for a in line[0].split()) for line in Qp))

To get x, y, z (and Weight) as columns of a dataframe:

>>> import pandas
>>> import re
>>>
>>> with open('out.test','r') as f:
...     for i, line in enumerate(f,1):
...         m = re.search('MESH : (\d+) unique points', line)
...         if m:
...             break
...
>>> i
6
>>> m.group(1)
'4'
>>> df = pd.read_csv('out.test', skiprows=i, nrows=int(m.group(1))+1, sep=r"\s+")
>>> df
       x    y    z  Weight
1  0.000  0.0  0.0  0.3906
2  0.125  0.0  0.0  0.7812
3  0.250  0.0  0.0  0.7812
4  0.375  0.0  0.0  0.7812
user_144210350
Related