Use StringIO as stdin with Popen

user_324969 0
Viewed 15265

I have the following shell script that I would like to write in Python (of course grep . is actually a much more complex command):

#!/bin/bash

(cat somefile 2>/dev/null || (echo 'somefile not found'; cat logfile)) \
| grep .

I tried this (which lacks an equivalent to cat logfile anyway):

#!/usr/bin/env python

import StringIO
import subprocess

try:
    myfile = open('somefile')
except:
    myfile = StringIO.StringIO('somefile not found')

subprocess.call(['grep', '.'], stdin = myfile)

But I get the error AttributeError: StringIO instance has no attribute 'fileno'.

I know I should use subprocess.communicate() instead of StringIO to send strings to the grep process, but I don't know how to mix both strings and files.

subprocess python
3 Answers

The following answer uses shutil as well --which is quite efficient--, but avoids a running a separate thread, which in turn never ends and goes zombie when the stdin ends (as with the answer from @jfs)

import os 
import subprocess
import io
from shutil import copyfileobj

file_exist = os.path.isfile(file)
with open(file) if file_exists else io.StringIO("Some text here ...\n") as string_io:
    with subprocess.Popen("cat", stdin=subprocess.PIPE, stdout=subprocess.PIPE, universal_newlines=True) as process:
        copyfileobj(string_io, process.stdin)
        # the subsequent code is not executed until copyfileobj ends, 
        # ... but the subprocess is effectively using the input.

        process.stdin.close()  # close or otherwise won't end

        # Do some online processing to process.stdout, for example...
        for line in process.stdout:
            print(line) # do something

Alternatively to close and parsing, if the output is known to fit in memory:

        ...
        stdout_text , stderr_text = process.communicate()
user_48589260
Related