Ensuring subprocesses are dead on exiting Python program

You can use atexit for this, and register any clean up tasks to be run when your program exits.

atexit.register(func[, *args[, **kargs]])

In your cleanup process, you can also implement your own wait, and kill it when a your desired timeout occurs.

>>> import atexit
>>> import sys
>>> import time
>>> 
>>> 
>>>
>>> def cleanup():
...     timeout_sec = 5
...     for p in all_processes: # list of your processes
...         p_sec = 0
...         for second in range(timeout_sec):
...             if p.poll() == None:
...                 time.sleep(1)
...                 p_sec += 1
...         if p_sec >= timeout_sec:
...             p.kill() # supported from python 2.6
...     print 'cleaned up!'
...
>>>
>>> atexit.register(cleanup)
>>>
>>> sys.exit()
cleaned up!

Note -- Registered functions won't be run if this process (parent process) is killed.

The following windows method is no longer needed for python >= 2.6

Here's a way to kill a process in windows. Your Popen object has a pid attribute, so you can just call it by success = win_kill(p.pid) (Needs pywin32 installed):

    def win_kill(pid):
        '''kill a process by specified PID in windows'''
        import win32api
        import win32con

        hProc = None
        try:
            hProc = win32api.OpenProcess(win32con.PROCESS_TERMINATE, 0, pid)
            win32api.TerminateProcess(hProc, 0)
        except Exception:
            return False
        finally:
            if hProc != None:
                hProc.Close()

        return True

On *nix's, maybe using process groups can help you out - you can catch subprocesses spawned by your subprocesses as well.

if __name__ == "__main__":
  os.setpgrp() # create new process group, become its leader
  try:
    # some code
  finally:
    os.killpg(0, signal.SIGKILL) # kill all processes in my group

Another consideration is to escalate the signals: from SIGTERM (default signal for kill) to SIGKILL (a.k.a kill -9). Wait a short while between the signals to give the process a chance to exit cleanly before you kill -9 it.

The subprocess.Popen.wait() is the only way to assure that they're dead. Indeed, POSIX OS's require that you wait on your children. Many *nix's will create a "zombie" process: a dead child for which the parent didn't wait.

If the child is reasonably well-written, it terminates. Often, children read from PIPE's. Closing the input is a big hint to the child that it should close up shop and exit.

If the child has bugs and doesn't terminate, you may have to kill it. You should fix this bug.

If the child is a "serve-forever" loop, and is not designed to terminate, you should either kill it or provide some input or message which will force it to terminate.

Edit.

In standard OS's, you have os.kill( PID, 9 ). Kill -9 is harsh, BTW. If you can kill them with SIGABRT (6?) or SIGTERM (15) that's more polite.

In Windows OS, you don't have an os.kill that works. Look at this ActiveState Recipe for terminating a process in Windows.

We have child processes that are WSGI servers. To terminate them we do a GET on a special URL; this causes the child to clean up and exit.

poll( )

Check if child process has terminated. Returns returncode attribute.

Is there a way to ensure all created subprocess are dead at exit time of a Python program? By subprocess I mean those created with subprocess.Popen().

You could violate encapsulation and test that all Popen processes have terminated by doing

subprocess._cleanup()
print subprocess._active == []

If not, should I iterate over all of the issuing kills and then kills -9? anything cleaner?

You cannot ensure that all subprocesses are dead without going out and killing every survivor. But if you have this problem, it is probably because you have a deeper design problem.

You can try subalive, a package I wrote for similar problem. It uses periodic alive ping via RPC, and the slave process automatically terminates when the master stops alive pings for some reason.

https://github.com/waszil/subalive

Example for master:

from subalive import SubAliveMaster

# start subprocess with alive keeping
SubAliveMaster(<path to your slave script>)

# do your stuff
# ...

Example for slave subprocess:

from subalive import SubAliveSlave

# start alive checking
SubAliveSlave()

# do your stuff
# ...

It's possible to get some more guarantees on windows by spawning a separate process to oversee the destruction.

import subprocess
import sys
import os

def terminate_process_on_exit(process):
    if sys.platform == "win32":
        try:
            # Or provide this script normally. 
            # Here just to make it somewhat self-contained.
            # see https://stackoverflow.com/a/22559493/3763139
            # see https://superuser.com/a/1299350/388191
            with open('.process_watchdog_helper.bat', 'x') as file:
                file.write(""":waitforpid
tasklist /nh /fi "pid eq %1" 2>nul | find "%1" >nul
if %ERRORLEVEL%==0 (
    timeout /t 5 /nobreak >nul
    goto :waitforpid
) else (
    wmic process where processid="%2" call terminate >nul
)""")
        except:
            pass
        
        # After this spawns we're pretty safe. There is a race, but we do what we can.
        subprocess.Popen(
            ['.process_watchdog_helper.bat', str(os.getpid()), str(process.pid)],
            stdout=subprocess.DEVNULL,
            stderr=subprocess.DEVNULL
        )

# example
class DummyProcess:
    def __init__(self, pid):
        self.pid = pid
set_terminate_when_this_process_dies(DummyProcess(7516))

This is what I did for my posix app:

When your app exists call the kill() method of this class: http://www.pixelbeat.org/libs/subProcess.py

Example use here: http://code.google.com/p/fslint/source/browse/trunk/fslint-gui#608

help for python code: http://docs.python.org/dev/library/subprocess.html#subprocess.Popen.wait