Detach command from SSH session

Before using special tools

1. Undertanding

   Why do ssh not return when I try to background job on target.

   ssh user@host 'sleep 300 &'
   

   Then you have to hit Ctrl+C to get your session.

   This is because each connection hold 3 file descriptors: STDIN (0), STDOUT (1) and STDERR (2). The connection will stay open while at least one of this file descriptors are in use.

2. Simply closing them before running background task

   This is aproximatively what nohup do.

   ssh user@host 'exec 0<&-;exec 1>&-;exec 2>&-; sleep 300 &'
   

   This will do the job.

3. Redirecting output to static file(s)

   You could store output in files located somewhere on server side:

   ssh user@host 'exec 0<&-;exec 1>>/path/to/logfile;exec 2>>/path/to/errlog; sleep 300 &'
   

   >> for adding log to existing files... you could use uniq files by adding $$ or date +%F...

   ssh user@host 'exec 0<&-;exec 1>/path/to/logfile-$$;exec 2>&1; sleep 300 &'
   

   2>&1 cited after 1>file will redirect STDERR as STDOUT to file.

   ssh user@host 'exec 0<&-;exec 1>/path/to/file-$(date +%F-%T).$$;exec 2>&-;sleep 300 &'
   

   Only STDOUT will be stored in newfiles.

Using screen

4. If screen is installed

   If you would be able to access interactive front end, you could:

   ssh user@host screen -dmS mySleep sh -c \
             "'x=0;while [ \$x -lt 300 ];do echo \$x;x=\$((x+1));sleep 1;done'"
   

   This will start a background job, printing 1 ling each 300 next second, then finish.

   To reconnect this, you could

   ssh -t user@host screen -x mySleep
   

   Then Ctrl+A, followed by d for leaving background task.

5. Installing screen at user level

   If you can't install something in server, maybe** could you install something in your $HOME directory:

   (** This will be easy if your architecture is same than on server. If not, you even could find compatible binaries or cross compile yourself.)

    $ ssh user@host
    user@host:~$ mkdir bin lib
    user@host:~$ exit
    $ scp /path/to/bin/screen user@host:~/bin
    $ ssh user@host
    user@host:~$ bin/screen
    bin/screen: error while loading shared libraries: libutempter.so.0:
              cannot open shared object file: No such file or directory
    user@host:~$ exit
    $ scp /path/to/lib/libutempter.so.0 user@host:~/lib
    $ ssh user@host
    user@host:~$ LD_LIBRARY_PATH=~/lib bin/screen
    bin/screen: /lib/x86_64-linux-gnu/libcrypt.so.1: version `XCRYPT_2.0' 
              not found (required by bin/screen)
    user@host:~$ exit
    $ scp /lib/x86_64-linux-gnu/libcrypt.so.1  user@host:~/lib
    $ ssh user@host
    user@host:~$ LD_LIBRARY_PATH=~/lib bin/screen
    Cannot make directory '/run/screen': Permission denied
    user@host:~$ mkdir $HOME/.screen
    user@host:~$ SCREENDIR=$HOME/.screen LD_LIBRARY_PATH=~/lib bin/screen
   

   Finally, now it work...

    $ ssh user@host SCREENDIR=\~/.screen LD_LIBRARY_PATH=\~/lib bin/screen -dmS mySleep sh -c \
             "'x=0;while [ \$x -lt 300 ];do echo \$x;x=\$((x+1));sleep 1;done'"
    $ ssh -t user@host SCREENDIR=\~/.screen LD_LIBRARY_PATH=\~/lib bin/screen -x mySleep
   

There are lot of other alternatives, like tmux, byobu...

Upto create your own wrapper...

nohup is the best tool for this, in my opinion.

Simply use it like:

nohup command &

As taken from nohup's man page:

If standard input is a terminal, redirect it from /dev/null. If standard output is a terminal, append output to 'nohup.out' if possible, '$HOME/nohup.out' otherwise. If standard error is a terminal, redirect it to standard output. To save output to FILE, use 'nohup COMMAND > FILE'.

You can use the screen or tmux utilities for this, if one is installed.

Have a look at reptyr. It is a utility for taking an existing running program and attaching it to a new terminal. You just need to start a screen, use reptyr to grab it, and then kill the ssh session. reptyr works by attaching to the target program using ptrace(2), redirecting relevant file descriptors, and changing the program's controlling terminal. After attaching a program, the program will appear to be either backgrounded or suspended to the shell it was launched from (depending on the shell). It is preferable to use bg; disown in the old shell to remove the association with the program, but reptyr will attempt to ensure that the target program remains running even if you close the shell without doing so.