ProcessPoolExecutor doesnt kill forked python process even after its completion

Viewed 8

I am creating ProcessPoolExecutor and then submitting some tasks to be processed. ProcessPoolExecutor forks multiple processes to do the task. The issue I am facing is some of the forked processes are stuck with their tasks even after the ProcessPoolExecutor is shut down. So when I see a process from the command prompt, I can see multiple processes with the same COMMAND

executor = ProcessPoolExecutor()
futures = [executor.submit(self.create_models, a, b) for col in self.df.columns]
        for future in as_completed(futures):

            try:
                model, col = future.result()
                models[f'm_{col}'] = model
            except Exception:
                logger.exception('Unable to get the results')
                raise
        executor.shutdown(wait=True)
0 Answers
Related