subprocess.Popen fails when called from within unittest, but works when called from __main__

user_535782 0
Viewed 226

I have a program that lauches processes with subprocess.Popen, it works when called normaly, but not within unittest (the python unit test framework).

python tests.py 
/usr/lib/python3.8/subprocess.py:942: ResourceWarning: subprocess 434608 is still running
  _warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
..
----------------------------------------------------------------------
Ran 2 tests in 0.002s


class BaseTests(unittest.TestCase):

It even fails when the process does nothing fancy, as in the following test case:

    def testPopen(self):

        subprocess.Popen(
            ['bash', '-c', "script-with-infinite-loop.sh"],
            stdout=subprocess.DEVNULL,
            stderr=subprocess.DEVNULL
        )


if __name__ == '__main__':


    unittest.main()

Note1: the purpose of the program is to launch process in the background, that are detached from the Python program. So waiting for the process to end would defeat the purpose.

Update

The detached process launching works with the following code:

with subprocess.Popen(
        ["nohup bash -c script-with-infinite-loop.sh &"],
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
        shell=True) as p:

    p.wait()

    if p.returncode != 0:
        raise Exception("failed")

But fails silently when running under unittests, i.e. the process returns with p.returncode == 0, but it seems that script-with-infinite-loop.sh never gets launched. I can see it because within script-with-infinite-loop.sh I log to a file (with echo 'sometthing'> f) , and the file is never created.

python-unittest python-3.x python
1 Answers

you never wait on the process, the unittest module enables warnings during tests to show you programming errors (such as zombie-ing a process like this)

the proper course of action is to wait on the process, either with .wait() or with .communicate() or to use the subprocess. run / call / check_call / check_output functions

alternatively, you could use the contextmanager form of Popen:

with subprocess.Popen(...):
    pass

if you'd like to see the same set of warnings during normal execution, try python -Wonce yourfile.py

user_8121830
Related