Point your OpenAI client at a model that can't read your prompts — and check that before the first request
By Nukri Basharuli and George Zhgenti Here's a setup we see a lot. A claims team at an insurer has a working prototype on the OpenAI SDK that writes a short summary of each claim file. Then legal asks who else can see t
By Nukri Basharuli and George Zhgenti
Here's a setup we see a lot. A claims team at an insurer has a working prototype on the OpenAI SDK that writes a short summary of each claim file. Then legal asks who else can see the claim text while the model is running. For any hosted API, the honest answer is the provider and whoever operates the GPUs, at least technically. The contract and the retention policy say they won't look. You have no way to check.
This post shows how to move that prototype to Confidential Router with almost no code changes, and how to verify where your data is going before every request.
The code change
Router speaks the standard OpenAI API. In Python you change base_url:
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8787/v1", # local Gatekeeper
api_key="sk-tee-v1-...",
)
resp = client.chat.completions.create(
model="<model-id>", # from GET /v1/models
messages=[{"role": "user", "content": "Summarize this claim file: ..."}],
)
print(resp.choices[0].message.content)
Notice the URL points to localhost, not to our servers. That's Gatekeeper, a small local process that checks the remote endpoint first and only then lets traffic through. Streaming, tool calling and /v1/embeddings (on models that support it) work the way you'd expect.
Install Gatekeeper
It's a single static binary with no dependencies, for Linux, macOS and Windows:
curl -fsSL https://github.com/Super-Protocol/confidential-router/releases/latest/download/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"
gatekeeper init
gatekeeper endpoint add router --upstream https://api.router.superprotocol.com --listen 127.0.0.1:8787
gatekeeper endpoint trust add router --from-upstream
gatekeeper run
trust add pins one specific deployment. It writes that deployment's evidenceDigest to your config: a SHA-256 over a snapshot of everything deployed, with container images pinned by digest. If we ship a new version, Gatekeeper stops forwarding requests until you approve the new digest yourself. In a regulated environment that's the behavior you want: a change to how your data is processed goes through you.
What gets checked
Before the first request, and every five minutes after that, Gatekeeper runs six stages: fetch, cert-chain, untrusted-root, jws, tls-fingerprint, policy. Two of them do most of the work.
The root of the certificate chain. The root CA runs inside a confidential VM on AMD SEV-SNP or Intel TDX. Gatekeeper checks that:
- the hardware report is signed by the CPU vendor;
- the report is bound to this CA's key;
- the launch measurement matches one rebuilt from our published build artifacts;
- that measurement is signed by a Super Protocol key compiled into the Gatekeeper binary.
The last check is there because without it, any other tenant on the same hardware would pass. All of this runs offline.
The TLS binding. The signed evidence bundle includes the fingerprint of the endpoint's TLS certificate. Gatekeeper compares it with the certificate it actually saw on the same connection. If something sits between you and the server, the fingerprints won't match.
If any stage fails, your app gets a 503 with an OpenAI-shaped error object and a stage field, and the request goes nowhere. Handle that case on its own instead of blindly retrying:
import openai
try:
resp = client.chat.completions.create(model=MODEL, messages=msgs)
except openai.APIStatusError as e:
if e.status_code == 503:
# Gatekeeper refused: evidence changed or a check failed.
# Don't fall back to an unverified endpoint.
alert_oncall(e.response.json())
raise
On successful responses Gatekeeper adds an X-Gatekeeper-Verdict: allow header. Log it next to the request ID in your audit trail.
Checking it yourself, without Gatekeeper
The evidence bundle is public and needs no key: https://api.router.superprotocol.com/.well-known/swarm-evidence. The simplest thing you can verify by hand is the TLS fingerprint:
import ssl, socket, hashlib, base64, json, urllib.request
HOST = "api.router.superprotocol.com"
def fp(der: bytes) -> str:
return "sha256/" + base64.urlsafe_b64encode(
hashlib.sha256(der).digest()).rstrip(b"=").decode()
bundle = json.load(urllib.request.urlopen(
f"https://{HOST}/.well-known/swarm-evidence"))
ctx = ssl.create_default_context()
with socket.create_connection((HOST, 443)) as s, \
ctx.wrap_socket(s, server_hostname=HOST) as tls:
seen = fp(tls.getpeercert(binary_form=True))
print("claimed :", bundle["certFingerprint"])
print("observed:", seen)
print("match :", seen == bundle["certFingerprint"])
When we ran this from our own build environment, the fingerprints didn't match. All outbound traffic there goes through a TLS-inspecting proxy, so the script saw the proxy's certificate. The bundle itself was intact: the hash of its embedded tlsLeaf matched the claimed fingerprint. If your corporate network does SSL inspection, Gatekeeper will deny at the tls-fingerprint stage. Add the router host to your proxy's bypass list; don't loosen the check.
The script is weaker than Gatekeeper: it opens two separate connections and doesn't verify the signature or the root. Use it to understand the mechanics and debug your network, not in production.
Where the check stops
Gatekeeper proves three things: which code is running, that it's running on genuine TEE hardware, and that you're connected to it. It doesn't currently verify GPU attestation. The docs list that as out of scope, even though inference itself runs on GPUs in confidential mode. If you call https://api.router.superprotocol.com/v1 directly and skip Gatekeeper, nothing is verified. For tools that only accept https endpoints that's the only option, so know what you're giving up.
The operator sees billing metadata only: model, token counts, timing. Each generation record (GET /v1/generation) carries the evidence_digest of the deployment that served it, and never any content.
Back to legal
Now the answer to legal's question isn't a pointer to the contract. The claim text is decrypted only inside an attested environment. Before every request, a process we run ourselves confirms it's the deployment we approved, and blocks the request if anything changed. The hash of that deployment sits in our log next to each call.
Router access is invite-only for now: superprotocol.com/developers/router. Docs: docs.router.superprotocol.com.
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.