Just to clarify that we're talking about the same thing in case I misunderstood something: autossh (style) scripts do these things:
1. fake data to keep a connection "fresh" for shitty middleware
2. detect connection which are stuck (state = open, but no data can actually round trip) and kill them
3. restart ssh when that happens
Is that what we're talking about here? I think people are saying that points 1 and 2, but not 3, are covered by SSH's ServerAlive* options. And that's also how OpenSSH advertises and documents those options, and apparently even how autossh talks about it in their own readme.
You're saying that those options don't actually solve points 1 and 2, while (your/their/etc) autossh does properly detect it.
Correct so far?
If so that seems like a bug in OpenSSH (or whatever implementation) which should get appropriate attention upstream. Has anyone reported this upstream? Is there a ticket to follow?
PS: I think we're all in agreement that option 3 is out of scope for stock OpenSSH (regardless of what other tools do)
I haven’t revisited this issue in years but on a project for thousands of similar devices we found autossh much more reliable.
I believe the issue is that the connections often fail or get wedged in other network layers; the only way to be sure that your ssh tunnel isn’t: a) lossy enough to “keep alive” but too lossy to send data, or b) isn’t just always waiting on TCP retry backoff, or c) etc, is to use the tunnel to transmit actual data at the application level.
> is to use the tunnel to transmit actual data at the application level.
Isn't that exactly what ServerAliveInterval does? The man page says: "ssh(1) will send a message through the encrypted channel". A plain TCP keepalive wouldn't count as being "through the encrypted channel".
I don't know if I would call it smoke and obfuscation, at the time systemd was not widely deployed and the ssh functionality was not as developed, so it made sense to use autossh. Now it sounds like it doesn't make sense anymore. It happens.
You summarized things well. #2 is the primary reason that ssh in a loop doesn't work as well or as reliably as autossh (the program discussed here; it's just coincidental that my own automatic ssh script is also called autossh).
1. fake data to keep a connection "fresh" for shitty middleware
2. detect connection which are stuck (state = open, but no data can actually round trip) and kill them
3. restart ssh when that happens
Is that what we're talking about here? I think people are saying that points 1 and 2, but not 3, are covered by SSH's ServerAlive* options. And that's also how OpenSSH advertises and documents those options, and apparently even how autossh talks about it in their own readme.
You're saying that those options don't actually solve points 1 and 2, while (your/their/etc) autossh does properly detect it.
Correct so far?
If so that seems like a bug in OpenSSH (or whatever implementation) which should get appropriate attention upstream. Has anyone reported this upstream? Is there a ticket to follow?
PS: I think we're all in agreement that option 3 is out of scope for stock OpenSSH (regardless of what other tools do)