What a reverse SSH tunnel leaves open
A reverse SSH tunnel is a neat answer to a real problem. The machine you need to reach sits behind a firewall you do not control, so you have it dial out to a server you do control and keep that connection open. When you
A reverse SSH tunnel is a neat answer to a real problem. The machine you need to reach sits behind
a firewall you do not control, so you have it dial out to a server you do control and keep that
connection open. When you need a shell, you go through the server and down the tunnel.
It works, and for one or two machines it is a reasonable thing to build. But the server in the
middle quietly becomes part of every device's trust story. This post walks through what that
server can leave open, and what we did differently in RescueShell. We describe built design and
what the security page says. We have not run a hostile-relay exercise against it and we do not
claim one.
A port that listens
The usual reverse tunnel needs a listening port somewhere: on the jump host, or forwarded on the
device or a relay. Every listening port is something to scan, rate-limit and patch.
Our agent dials out over HTTPS, on port 443, and the device opens no port. The installer refuses a
relay address that does not start with https://. Nothing connects in to the customer's network
(apps/site/src/pages/security.html, how-it-works.html).
A jump host that is a skeleton key
This is the one that matters most. In a plain reverse tunnel, the jump host decides which keys
reach which devices, or at least sits where it can change that decision. Whoever controls it
controls every device behind it: add a key, and you have a shell.
We moved that decision onto the device. Each device keeps its own list of trusted keys and
accounts. Our servers cannot add a key or an account to it. A new install has to be given that
list and refuses to run without it. So if our servers were compromised, they still could not add a
key or an account on a device.
One gap in our earlier design is now closed, and the threat model keeps it listed as such. Before
the device started asking for a signature, the relay used to name a trusted key and an account, and
the agent would open a pipe to the device's sshd for it. Public keys are
not secret, so a hostile relay could have got that pipe. It could not have authenticated, but it
would have reached sshd's pre-auth surface.
Now the agent answers a new session with a fresh nonce and nothing else. It writes a key and
dials sshd only when the client returns an SSH signature, in the namespace
rescueshell-session-v1, over a payload that names this device, this session, the account, the
host key and the nonce. The key that signs must already be in the device's own trust list. Our
servers hold no key, so they cannot answer, and a signature made for one session cannot be reused
for another (docs/security/threat-model.md, docs/adr/0030-*).
The shell itself is end to end. In the browser, SSH runs in your browser, in a separate signed frame, with your own key, which is
either loaded for one session or saved encrypted by your passkey. With the CLI, native ssh runs
with your own key and the private key never leaves your laptop. The relay forwards encrypted bytes
and cannot read the session. The agent refuses password sessions.
The goal we hold ourselves to is stated in the threat model: full control of our backend and relay
should open zero shells and yield zero key material. The properties above are built and are what the
security page lists. The red-team round that would test the goal end to end has not been run, so we
call it a goal, not a proven result.
What stays open
We would rather list the residuals than leave them for you to find.
- A backend or relay owner can still refuse or drop every session. That is availability, not a shell.
- They can still see metadata: who connected where, and when.
- A new install fetches
install.shfrom our origin, so a backend owner could serve a bad script to a device being installed at that moment. The agent tarball is signed by a key kept out of the backend. Devices that are already installed are not affected by a swapped script. - A compromised web app can still show a fake form that asks you to paste a key. Saved keys and the CLI are the answers to that.
Keys that never get rotated
Reverse tunnels tend to collect long-lived keys: one shared key on the jump host, copied onto every
device, that nobody dares rotate. Key sprawl is how an old laptop keeps access for years.
We hold no shared keys. Each provisioning credential belongs to one team, and refresh tokens are
stored hashed. Adding or removing a key on a device is a trust change that has to be signed by a
key the device already trusts (namespace rescueshell-trust-v1). Our server stores and relays the
change and never signs it. From the browser or the CLI you can sign one batch for a group of
devices, so a rotation is one step and not a visit to each machine.
For enrolling many devices at once, the enrolment secret is made and kept by your client. The
server never sees it.
Trust on first use, with the blind spot filled
The first time you connect to a device, SSH asks you to accept its host key. In a tunnel, that
first connection often goes through the jump host, so the party you are trusting to tell you the
key is the party you are worried about.
We do not pin a host key on the relay's word alone. The CLI and the browser pin a device's host key
on first connect only on evidence the relay cannot supply. Either you paste the fingerprint the
device printed (from the installer or rescueshell-agent --status), and it is compared whole, or
the device attests its host key with a key derived from an enrolment secret that your client made
and holds. After that, a change is refused. The relay also records the key it first sees and
refuses shells if it changes, until an admin accepts the new key.
One residual, kept short: the attested path is weaker than a paste if the enrolment secret leaks,
because the devices of one enrolment pass share that secret. A paste depends on no shared secret.
A session that is only a key
A plain reverse SSH session is "the client holds a key that the jump host or sshd accepts". Steal
the key, or the laptop it lives on, and you have the shell.
Opening a shell in RescueShell takes more than the key. A new team is asked to set up a passkey or
authenticator app before its first device, and every new shell asks for it again: in the browser a
passkey or an authenticator code, in the CLI an authenticator code or a recovery code. Then the
device itself wants a fresh signature from a key it already trusts before it opens a pipe to sshd,
as above. If you want a third step, install.sh --ask-password makes sshd ask the account password
after the session key.
The security page also lists optional two-factor login with an authenticator app and single-use
recovery codes. That is a separate, account-level feature from the per-shell prompt.
An agent that never gets updated
The quiet failure of any fleet is the old version nobody got round to replacing. Until recently, a
new agent version reached a device only through a signed release and a local install. Since agent
0.2.3, the agent can update itself, and the way it does so is built to keep the same promise.
The trust root is the release signing key, the same one install.sh pins, baked into the binary at
build time. The relay is a courier. It may say that a team allows updates and it serves the files,
but every byte is accepted only after SHA256SUMS.sig verifies against the baked key and the
artifact's hash matches the signed SHA256SUMS. Any other outcome leaves the running agent
untouched. The previous binary is kept as .prev and restored if the new one never holds a relay
connection. Updates run about once a day, and only when no shell is open.
A team can turn automatic updates off in its settings, and a single device can lock itself off with
install.sh --no-auto-update. An update changes the code, never who the device trusts.
Devices on an older agent need one manual install of 0.2.3 before they can update themselves.
Where this does not fit
The design buys safety by putting work on the device and on you. The trust list has to be given at
install time. A shell needs a key you hold and a second-factor prompt each time. That is
more steps than pasting a password into a jump host, and we think it is worth it, but it is a
real cost if your fleet is a handful of machines on a bench.
It also does not make the service unkillable. If our servers are down or hostile, you cannot get
a shell through them. We do not offer an uptime SLA or high availability, and we list that on the
security page.
RescueShell: €0.10 a device a month, billed yearly, at least €6 a year, plus VAT. Thirty-day
trial, card required, and the clock starts when your first device connects. Hosted in the EU,
exceptions listed.
Try it
First published on the RescueShell blog.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.