Extract a specific text in between 2 characters in a line from a text file using Python

Your file is in the csv format. Python has a built-in library for helping read csv's: https://docs.python.org/3/library/csv.html

I just printed out the results but of course, you could write the results to a file.

import csv

with open(file_path, "r") as f:
    reader = csv.reader(f, delimiter=',')

    for row in reader:
        print(row[0], row[11], row[39])

You can also provide column headers to avoid the ugly numbers to access the correct columns. See https://docs.python.org/3/library/csv.html#csv.DictReader for details.

If you want additional features and to do more manipulation with the data, Pandas is a great library for this type of data. https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html would help you get the data in.

Probably the simplest manner to do this is using the .split() method of strings. In each iteration of that for, you can use data=line.split(',') to break the entire line into several "segments", each one between a pair of commas or the end of line (you can substitute the ',' for any other char/string to break lines according to other chars).

Using this approach, data[0] would contain the time/date, data[1] would contain the string "General", data[2] would contain "connected?", and so on.

Finally, you just need to write down the data[] using the proper indices as follows: outFile.write(data[0] + "," + data[1] + "," + data[2])

Just remember to verify the index of the columns you want to save based on the input file, and use the proper index when writing.

Since you did add the powershell tag on your question and I removed it (feeling guilty) I guess I could show you how this could be done using PS.

This will result in an object that you can easily manipulate, for example:

PS /> $result.'Date and Time'
2021.08.09.11.32.35.7795
2021.08.09.11.32.45.7868
2021.08.09.11.32.51.0936
2021.08.09.11.32.55.8040

PS /> $result.'Date and Time'[2]
2021.08.09.11.32.51.0936

PS /> $result.'temperature in deg C'
26
26
26
26

PS /> $result.Where({$_.'Laser Head Temperature in deg C' -eq 26}) | Select-Object 'Date and Time','Laser Head Temperature in deg C'

Date and Time            Laser Head Temperature in deg C
-------------            -------------------------------
2021.08.09.11.32.35.7795 26                             
2021.08.09.11.32.51.0936 26                             
2021.08.09.11.32.55.8040 26      

The code would look something like this:

$raw = @'
2021.08.09.11.32.35.7795, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 37, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 42, Laser Head Temperature in deg C, 26, Two Photon Diode in mV, 12, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)
2021.08.09.11.32.45.7868, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 34, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 42, Laser Head Temperature in deg C, 25, Two Photon Diode in mV, 11, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)
2021.08.09.11.32.51.0936, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 37, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 42, Laser Head Temperature in deg C, 26, Two Photon Diode in mV, 11, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)
2021.08.09.11.32.55.8040, General, connected?, connected, operation state, connecting, laser state, Standby, pulse picker trigger, internal, Amplifier LD current in mA, 0, PP Frequency in kHz, 10000, System Reset, 65535, Bust Mode, 1, Modelock, no, Oscillator Ready, not ready, Front-end Ready, ready, current in mA, 0, temperature in deg C, 26, power in mW, 34, Pulse Picker Frequency in kHz, 0, Laser ON Runtime in hour, 1894, Laser Head Humidity in %, 41, Laser Head Temperature in deg C, 26, Two Photon Diode in mV, 11, Laser Power in mW, 34, Error Code, 0, Laser FW Version, 7.4.6, TcpModbus protocol version, 1.1.4, state, Null, On, FALSE, Good, FALSE, Fast Blank, FALSE, Treating, FALSE, Error, FALSE, Error Detail, (No Error)
'@ -split '\r?\n'

$i=0

$result = $raw | ForEach-Object {

    $arr = $_ -split ',' -ne ''
    $col, $val = $arr[2..($arr.Length-1)].where({
        if($i = -not $i){$_}
    },'Split')
    
    $out = [ordered]@{}
    $out.Add('Date and Time',$arr[0].Trim())
    $out.Add('General?',$arr[1].Trim())

    for($z=0;$z -lt [math]::Max($col.Count,$val.Count);$z++)
    {
        $out.Add($col[$z].Trim(),$val[$z].Trim())
    }

    [pscustomobject]$out
}

Note: The $raw variable is there just to test the code, however, since you're reading a .log file, you should read it using Get-Content:

$raw = Get-Content 'absolute/path/to/Laser-20210809-113225.log'

Lastly, this is how the $result object looks like when using Out-GridView:

Finally this worked, thanks all for your help, really appreciate:

import pandas as pd
import csv
import time
import datetime

pd.set_option('display.max_rows',10000)

timestr = time.strftime("%Y%m%d-%H%M%S")
file_start = r".\output_"
extension = ".csv"

file_name = file_start + timestr + extension

outFile = open(file_name, "w")
df = pd.read_csv(r'C:\Users\sthilla1\Downloads\2021-08-09\11-32-25\Laser-20210809-113225.csv', usecols = [0,25,26], low_memory = True)
outFile.write(str(df))
outFile.close()

For getting the date you can use regular expressions:

import re

text = "2021.08.09.11.32.35.7795 text text text 2001.02.01.12.35.55.7795"

regex_pattern = "[0-9]{4}(?:\.[0-9]{2}){5}\.[0-9]{4}"

date_list = re.findall(regex_pattern, text) #to get many ones

date = re.search(regex_pattern, text)[0] #to get only one

print(date_list)
print("-------")
print(date)

result:

['2021.08.09.11.32.35.7795', '2001.02.01.12.35.55.7795']
-------
2021.08.09.11.32.35.7795