terminate python multiprocessing pool cleanly

Question

I am using multiprocessing.pool to work with the http server in python - it works great, but when I terminate, I get a slew of errors from all the spawnpoolworkers - and I'm just wondering how I avoid this.

My main code:

    def run(self):        
        global pool
        port = self.arguments.port
        pool = multiprocessing.Pool( processes= self.arguments.threads)
        with http.server.HTTPServer( ("", port), Handler ) as daemon:
            print(f"serving on port {port}")
            while True:
                try:
                    daemon.handle_request()
                except KeyboardInterrupt:
                    print("
exiting")
                    pool.terminate()
                    pool.join()
                    return 0

I've tried doing nothing to the pool, I've tried doing pool.close() - I've tried not joining. But even if I just run that - never even access the port or call anything onto the pool, I still get a random list of things like this when I press control-c

Process SpawnPoolWorker-8:

Process SpawnPoolWorker-4:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/queues.py", line 365, in get
    with self._rlock:
  File "/opt/homebrew/Cellar/python@3.10/3.10.1/F

how do I exit the pool cleanly, with no errors, and with no output?

user_2446374 · Answer

ok - I'm stupid - the control-c was also interrupting all the child processes. This fixed it:


def ignore_control_c():
    signal.signal(signal.SIGINT, signal.SIG_IGN)

pool = multiprocessing.Pool( processes = self.arguments.threads, initializer = ignore_control_c )

terminate python multiprocessing pool cleanly

1 Answers