If your SSH session suddenly drops with the message:
|
1 2 |
client_loop: send disconnect: Broken pipe |
…you’ve hit a common—but fixable—network or keepalive issue. In this post we’ll show quick client-side remedies, server-side hardening, and a few diagnostics to pinpoint the root cause. Whether you’re connecting to a cloud VM, an on‑prem server behind a firewall, or a laptop on hotel Wi‑Fi, these steps will make your connection far more reliable.
TL;DR
Add keepalives to your SSH client and (optionally) your server. Start here:
|
1 2 |
ssh -o ServerAliveInterval=60 -o ServerAliveCountMax=3 user@host |
Make it permanent by adding this to ~/.ssh/config:
|
1 2 3 4 5 6 7 |
Host * ServerAliveInterval 60 ServerAliveCountMax 3 TCPKeepAlive yes # Helps on some networks that throttle low‑priority packets IPQoS 0 |
If you control the server, add:
|
1 2 3 4 5 |
# /etc/ssh/sshd_config ClientAliveInterval 60 ClientAliveCountMax 3 TCPKeepAlive yes |
Reload SSH: sudo systemctl reload sshd (or service ssh reload).
Why this happens
“Broken pipe” means your SSH client tried to send data but the underlying TCP connection was gone. Common reasons:
- Idle/NAT timeouts: Firewalls, load balancers, and carrier‑grade NATs often kill quiet TCP sessions.
- Flaky networks or roaming Wi‑Fi: Short drops sever long‑lived connections.
- Aggressive security appliances: Intrusion prevention may reset idle or long sessions.
- MTU/QoS oddities: Some networks deprioritize or fragment small keepalive packets.
- Rekey intervals: Infrequent, but if disconnects occur at clockwork intervals, rekey negotiation might be failing.
Quick client-side fixes
Start with temporary flags to confirm keepalives solve it. If they do, bake them into your config.
|
1 2 3 4 5 6 7 8 9 |
# Gentle keepalives (1 minute heartbeat, 3 missed heartbeats before exit) ssh -o ServerAliveInterval=60 -o ServerAliveCountMax=3 user@host # On unstable links, be a bit chattier ssh -o TCPKeepAlive=yes -o ServerAliveInterval=30 -o ServerAliveCountMax=6 user@host # If QoS is messing with your packets ssh -o IPQoS=0 user@host |
Make it permanent in ~/.ssh/config:
|
1 2 3 4 5 6 |
Host * ServerAliveInterval 60 ServerAliveCountMax 3 TCPKeepAlive yes IPQoS 0 |
Tip:
ServerAlive*are application-level keepalives.TCPKeepAliveleverages the OS stack. Using both increases your odds through finicky middleboxes.
Server-side hardening (if you’re the admin)
Give your users a more resilient baseline by enabling server keepalives and sensible TCP keepalive sysctls.
OpenSSH:
|
1 2 3 4 5 |
# /etc/ssh/sshd_config ClientAliveInterval 60 ClientAliveCountMax 3 TCPKeepAlive yes |
Reload sshd:
|
1 2 3 4 |
sudo systemctl reload sshd # or sudo service ssh reload |
Kernel TCP keepalives: (system-wide safety net)
|
1 2 3 4 5 6 7 8 9 10 |
# View current values sysctl net.ipv4.tcp_keepalive_time sysctl net.ipv4.tcp_keepalive_intvl sysctl net.ipv4.tcp_keepalive_probes # Reasonable defaults (apply via /etc/sysctl.d/*.conf) net.ipv4.tcp_keepalive_time = 600 # seconds before starting keepalives net.ipv4.tcp_keepalive_intvl = 60 # seconds between probes net.ipv4.tcp_keepalive_probes = 5 # number of failed probes before drop |
Firewalls & LBs: Increase stateful TCP idle timeouts (often 300–900s works well for SSH).
Fast diagnosis playbook
When it still drops, get a quick read on what’s failing.
- Verbose client logs
12ssh -vvv user@host
Look near the end for timeouts, rekey notices, or suddendisconnectmessages. - Server logs
- Debian/Ubuntu:
1234sudo journalctl -u sshd -f# and/orsudo tail -f /var/log/auth.log - RHEL/Alma/CentOS:
12sudo tail -f /var/log/secure
- Debian/Ubuntu:
- Pattern check
- Many sessions drop at once? Suspect upstream firewall/LB.
- Only idle shells die? Turn up
ClientAliveInterval/ServerAliveInterval. - Drops exactly every N minutes? Check firewall state timers or SSH rekey limits.
- Network sanity
- If you’re on VPN/coffee shop Wi‑Fi/hotel: expect aggressive NAT. Use short intervals (30–60s) and
IPQoS 0. - If large transfers die midway, prefer resilient tools.
- If you’re on VPN/coffee shop Wi‑Fi/hotel: expect aggressive NAT. Use short intervals (30–60s) and
Working around unstable links
- Use mosh for interactive shells over spotty networks. Mosh tolerates roaming, IP changes, and brief outages gracefully. (You’ll still use SSH for scp/rsync.)
- Use rsync wisely for big copies:
12rsync -av --partial --append-verify src/ user@host:/dest/
This resumes instead of restarting from zero. - Screen/Tmux: Always run long tasks inside
tmuxorscreenso a dropped session doesn’t kill your process. Reattach after reconnect:
1234tmux new -s work# ... do work ...tmux attach -t work
Less common culprits (but worth a try)
- MTU/fragmentation: If you’re traversing VPNs, test a lower MTU at the OS or VPN level.
- QoS deprioritization:
IPQoS 0(client) avoids DSCP markings that some networks punish. - Rekey intervals: If disconnects hit, say, every 60 minutes, align
RekeyLimitbetween client and server. Example on the server:
123# /etc/ssh/sshd_configRekeyLimit 512M 1h
A sensible default checklist
- Client: Add
ServerAliveInterval 60,ServerAliveCountMax 3,TCPKeepAlive yes,IPQoS 0. - Server: Enable
ClientAliveInterval 60,ClientAliveCountMax 3,TCPKeepAlive yes; reload sshd. - Network: Bump firewall/LB idle timeouts to ≥10 minutes for SSH.
- Workflows: Use
tmuxand resilient transfer tools (rsync --append-verify). - Still failing? Capture
ssh -vvvlogs and check server logs to isolate NAT vs. host problems.
Conclusion
“client_loop: send disconnect: Broken pipe” is frustrating, but the fix is rarely mysterious. Keepalives mitigate idle timeouts, small config tweaks smooth over finicky middleboxes, and resilient tools protect your work during the occasional network hiccup. Apply the client defaults, harden the server if you can, and you’ll turn random drops into a rare event.




