keepcalculator

06 Numbers and Powers

Keep Remote SSH Sessions Running After Disconnect

Card No. 06·83507 · Updated · Standard formula, shown below

To keep a remote SSH session running after you disconnect, use a terminal multiplexer like tmux or GNU Screen, or run the command with nohup for simple background execution. These tools detach your process from the SSH connection, so it continues running on the server even after you log off.

Why SSH Sessions Die on Disconnect

When you start a program over SSH, it runs as a child of the SSH session. The SSH server sends a SIGHUP (hangup) signal to all processes in the session when the connection closes—whether you type exit, lose network connectivity, or your laptop sleeps. This signal tells the processes to terminate, which is why a long-running command like wget or a data migration stops the moment your SSH link drops.

This behavior is intentional: it prevents orphaned processes from consuming server resources indefinitely. But for legitimate long-running tasks—training a model, compiling software, transferring large files—you need a way to detach the process from the terminal session.

Option 1: tmux (Recommended)

tmux is a terminal multiplexer that creates persistent sessions on the server. You start a session, run your command inside it, then detach. The session keeps running in the background, and you can reattach later from any SSH connection.

  1. Install tmux if not present: sudo apt install tmux (Debian/Ubuntu) or sudo yum install tmux (RHEL/CentOS).
  2. Start a new session: tmux new -s mytask
  3. Run your command inside the session (e.g., ./long_script.sh).
  4. Detach with Ctrl+b, then d. You’ll return to the normal shell.
  5. Reattach later with tmux attach -t mytask.

You can have multiple windows and panes within one session, and tmux keeps the session alive even if you close the terminal entirely. To list active sessions, use tmux ls.

Option 2: GNU Screen

GNU Screen is the older multiplexer with similar functionality. It’s often pre-installed on legacy systems. The workflow:

  1. Start a new screen session: screen -S mytask
  2. Run your command.
  3. Detach with Ctrl+a, then d.
  4. Reattach with screen -r mytask.

Screen is lighter and may be the only option on very minimal servers. However, tmux offers better scripting, split panes, and more active development.

Option 3: nohup for Simple Background Tasks

If you don’t need to interact with the process later, nohup runs a command immune to SIGHUP and redirects output to a file. For example:

nohup ./long_script.sh > output.log 2>&1 &

The & puts the job in the background, and nohup ensures it ignores the hangup signal. You can check progress by reading output.log. To stop the process, you must find its PID with ps aux | grep long_script and kill it.

This is the simplest approach for one-off commands where you don’t need to reattach. But you lose the ability to send input to the process, and debugging is harder.

How to Choose the Right Method

  • Need to check progress or interact later? Use tmux or screen.
  • Just need a background job that runs to completion? Use nohup.
  • Multiple parallel tasks? tmux windows/panes or separate screen sessions.
  • Server is minimal? Screen may be pre-installed; otherwise install tmux.

For any task lasting more than a few minutes, tmux is the safest bet because it also protects you from accidental disconnects (e.g., Wi-Fi drops).

Common Problems and Solutions

  • Forgot to start tmux and now the session is stuck? You can’t detach an existing process, but you can use reptyr to attach it to a tmux session—though it’s not always reliable. Better to start with tmux next time.
  • Session lost after server reboot? Neither tmux nor screen survives a reboot. For that, you need a process manager like systemd (create a service) or a tool like supervisord.
  • Output not appearing in nohup log? Ensure you redirect both stdout and stderr with 2>&1. Some programs buffer output; use stdbuf -oL to force line buffering.
  • Can’t reattach because the session is “dead”? If the server restarted, the session is gone. Check tmux ls or screen -ls to see if it exists.

Comparison: tmux vs. screen vs. nohup

Feature tmux screen nohup
Reattach to session Yes Yes No
Multiple windows Yes Yes No
Split panes Yes Yes (vertical only) No
Survives SSH drop Yes Yes Yes
Scriptable Excellent Good N/A
Learning curve Moderate Moderate Minimal

For most users, tmux is the best balance of features and ease of use. Screen is a fallback for older systems.

Keeping Sessions Alive with SSH Options

You can also configure SSH itself to reduce disconnections. In your local ~/.ssh/config, add:

Host myserver
  HostName example.com
  ServerAliveInterval 60
  ServerAliveCountMax 3

This sends a keepalive packet every 60 seconds, so your router or firewall doesn’t drop the idle connection. It doesn’t keep the session alive after a disconnect, but it prevents many timeouts.

On the server side, you can set ClientAliveInterval in /etc/ssh/sshd_config to the same effect. However, these are complementary to tmux—they reduce the chance of disconnection, but don’t protect against network drops.

Frequently Asked Questions

Can I keep a session alive without installing anything?

If you have no install permissions, you can use nohup (usually available) or setsid to run a process detached. For interactive reattachment, you’d need tmux or screen, but you can compile them in your home directory if needed.

What happens to my tmux session if the server reboots?

It’s lost. tmux does not persist across reboots. To run something at boot, create a systemd service or use cron @reboot.

Can I run multiple commands in one tmux session?

Yes, you can open new windows with Ctrl+b then c, and switch between them with Ctrl+b then n/p. Each window is an independent shell.

Is nohup safe for long-running scripts?

Yes, but you lose the ability to see progress interactively. You can monitor with tail -f output.log, and you must manage the process manually if it needs input.

How do I kill a detached process?

For tmux: tmux kill-session -t mytask. For screen: screen -X -S mytask quit. For nohup: find the PID with ps aux | grep command and kill PID.

Can I use tmux on Windows?

If you use Windows Subsystem for Linux (WSL) or PuTTY, you can install tmux on the Linux server and use it normally. The client-side OS doesn’t matter.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →