Make background subprocess output to single jupyter notebook cell

Viewed 96

In a Jupyter notebook I am using subprocess and IPython's BackgroundJobManager to create a background process. The next cells are then doing some other things while the background process is running. This is my code:

def _run_in_background(fn, *args, **kwargs):
    from IPython.lib import backgroundjobs as bg
    jobs = bg.BackgroundJobManager()
    jobs.new(fn, *args, kw=kwargs)


def _bash(cmd, print_stdout=True, print_stderr=True):
    import subprocess as sp
    import sys
    proc = sp.Popen(cmd, stderr=sp.PIPE, stdout=sp.PIPE,
                    shell=True, universal_newlines=True,
                    executable='/bin/bash')
    while proc.poll() is None:
        for stdout_line in proc.stdout:
            if stdout_line != '':
                if print_stdout:
                    print(stdout_line, end='')
        for stderr_line in proc.stderr:
            if stderr_line != '':
                if print_stderr:
                    print(stderr_line, end='', file=sys.stderr)
    return proc.wait()

_run_in_background(_bash, './myprogram')

The problem is, the program's output seems to be directed to the output of whatever cell is currently active, not the cell in which the background process was started (this seems logical since I simply redirected the process' stdout and stderr, nothing more fancy).

Is there a way to force the output of the process to be directed only to its cell?

0 Answers
Related