catch contents generated during run by subprocess in Python

user_13105995 0
Viewed 34

I would like to catch contents generated during the run instead of output since I realized that the output from program is unfortunately not the useful information.

basically my code:

command = "usearch11 -threads 8 -fastq_filter pmFLP.fastq -fastq_maxee 1  -fastq_minlen 120 -fastq_maxns 0  -fastaout S1--1.QC.fasta"
with Popen(command, stdout=PIPE, stderr=None, shell=True) as process:
    output = process.stdout.readlines()
    print(output)

and what is displayed in the terminal after/during running the code:

C:\Users\Desktop\Daten\test_Ordner>python test.py
00:00 4.6Mb  FASTQ base 33 for file pmFLP.fastq
00:01 5.3Mb   100.0% Filtering, 98.6% passed
    188650  Reads (188.7k)
      1297  Discarded read with > 0 Ns
      1132  Discarded reads with expected errs > 1.00
    185957  Filtered reads (186.0k, 98.6%)
b'usearch v11.0.667_win64, 137Gb RAM, 64 cores\r\n(C) Copyright 2013-18 Robert C. Edgar, all rights reserved.\r\nhttps://drive5.com/usearch\r\n\r\nLicense: kleine@planton.de\r\n\r\n'

and this part is what i need:

00:00 4.6Mb  FASTQ base 33 for file pmFLP.fastq
    00:01 5.3Mb   100.0% Filtering, 98.6% passed
        188650  Reads (188.7k)
          1297  Discarded read with > 0 Ns
          1132  Discarded reads with expected errs > 1.00
        185957  Filtered reads (186.0k, 98.6%)

any help will be appreciated!!

subprocess python
1 Answers

You problably need to catch stderr instead of stdout. Also I recommend using subprocess.run() not subprocess.Popen():

command = "usearch11 -threads 8 -fastq_filter pmFLP.fastq -fastq_maxee 1  -fastq_minlen 120 -fastq_maxns 0  -fastaout S1--1.QC.fasta"
completed_process = run(command, stderr=PIPE, shell=True, text=True)
desired_text = completed_process.stderr
print(desired_text)

If you want raw bytes instead, remove the text=True flag:

command = "usearch11 -threads 8 -fastq_filter pmFLP.fastq -fastq_maxee 1  -fastq_minlen 120 -fastq_maxns 0  -fastaout S1--1.QC.fasta"
completed_process = run(command, stderr=PIPE, shell=True)
desired_text = completed_process.stderr  # Those are now bytes, not str!
print(desired_text)

To write the output to a binary file:

command = "usearch11 -threads 8 -fastq_filter pmFLP.fastq -fastq_maxee 1  -fastq_minlen 120 -fastq_maxns 0  -fastaout S1--1.QC.fasta"

with open("summary.txt", "wb") as file:
    run(command, stderr=file, shell=True)

As you can see, you can also pass any writable file object to stderr or stdout.

user_35156700
Related