Dev.to Security 🔐 Cybersecurity 👁 0 📖 5 min read

AI agents don't break end-to-end encryption. They go around it.

I'm building Halonyx, an open-source Signal Protocol messenger. I started it to learn how X3DH and the Double Ratchet work from first principles. It isn't audited and it isn't for production. Lately I've been trying to w

I'm building Halonyx, an open-source Signal Protocol messenger. I started it to learn how X3DH and the Double Ratchet work from first principles. It isn't audited and it isn't for production. Lately I've been trying to work out what AI agents change for a project like this, and my answer is smaller and stranger than "AI breaks encryption."

It doesn't. Nothing about agents weakens X25519 or AES. What changes is where the weak point sits.

The message is safe. The screen isn't.

End-to-end encryption protects a message between two devices. Once it arrives, it is decrypted and shown on a screen, held in memory, possibly stored, and possibly echoed into a notification. An agent that runs on that device with broad permissions can read all of it without touching the cryptography.

I used to treat the endpoint as someone else's problem. I now think it belongs in the threat model, because the thing that reads your screen is increasingly a product feature that ships with the operating system, not malware.

Windows Recall is the clearest example. It screenshots the desktop every few seconds and builds a searchable database from the text. Any messenger open on that desktop ends up in the database. As far as I know, Signal Desktop responded by using a digital rights management flag that blocks screenshots. That works, but it also affects screen readers, which people with low vision rely on. I don't know a better option the app had. The point for me is that the available defense was blunt, and the cost landed on accessibility.

I'm a client developer, so that stuck with me. If I plan security and accessibility separately, an operating system change can force me to choose between them later.

Why agents want so much access

Take a task that sounds harmless: planning a friend's birthday dinner. To do it well, an agent needs message history to know who's coming, dietary restrictions, a calendar, a payment method, and a browser to make the booking. Each permission is reasonable on its own, and together they give the agent a lot.

The risky part is the browser. When the agent loads a restaurant's page, it reads text it didn't write and can't verify. That text can contain instructions, such as sending out data the agent can see. This is prompt injection, and as far as I know it's still unsolved. A multi-step task also means many steps where nobody is watching. A low-risk job (find a date) becomes a high-risk one because of what the agent has to hold in order to finish it.

Moving the model to a remote server doesn't fix this. You can encrypt the connection to the inference server perfectly well. The agent still needs your data to do its job, and that data still leaves your device.

The two-person problem

A chat has at least two people in it. If I give an agent broad access to my phone, I'm also giving it access to what the other person told me. They never agreed to that.

I think this is the strongest argument for treating agent access as a decision the app can weigh in on. The mechanism I'd want is simple:

  • Agent access to a messenger is off by default.
  • The app developer can declare that agents shouldn't read its data, and the operating system honors that. Without a mechanism like that, the app can't protect the other person in the conversation. It can only hope the user's settings are cautious.

Other ways AI changes the picture

Metadata gets more valuable. A human can't read millions of timestamps and contact pairs. A model can find patterns in them: who talks to whom, how often, when, and how that changes. Data that looked harmless when no one could process it looks different now. This is the part of Halonyx I'm least happy with, since the relay sees hashed sender and recipient IDs, timing and ciphertext size.

Impersonation gets cheaper. Voice cloning and generated messages make "it sounded like them" a weak signal. Safety Numbers and out-of-band verification matter more for that reason.

Scanning proposals get a new argument. Some governments want client-side scanning, and AI-generated abuse material is now used to argue that simple hash lists aren't enough. That pushes toward classifiers running on every device. Any scanning layer sits on the endpoint, outside the encryption, and whoever controls its rules controls what it flags. I don't think a narrow version stays narrow.

The platform can change underneath you. The operating system is controlled by a few companies, and an app can't stop them changing what the platform allows. A notification-storage bug on iOS, which was patched, showed how message content can end up somewhere the app never intended.

What I'm changing in Halonyx

Halonyx is a browser client, so it has fewer defenses than a native app, and I can't set an operating-system screenshot flag. None of the following is done. This is the list I'm working through.

  1. Add the agent case to the threat model. The STRIDE document already covers browser key storage and missing CSP. An agent or extension reading the page or storage is a separate threat, and the entry should say what Halonyx can't defend against.
  2. Say it in the README. End-to-end encryption doesn't protect a compromised endpoint. I'd rather state that plainly than let readers assume otherwise.
  3. Keep any assistant features local and narrow. If I ever add summaries or suggestions, a feature gets the minimum context it needs and no message content leaves the device.
  4. Be precise about the DLP scan. It checks outgoing messages for API keys, tokens and card numbers before encryption. That catches obvious leaks. It isn't a defense against prompt injection and I shouldn't describe it as one.
  5. Check hardening against accessibility. Any change that blocks screen capture or scraping gets tested with a screen reader first.
  6. Name an owner for AI-written code. Code in the crypto and key-handling paths needs a person who has read and understood every line, whoever or whatever drafted it. ## Where I'm unsure

I haven't seen evidence of an operating-system agent demanding access to a messenger's data in practice. What I'm describing is a direction, based on how these products are being built and marketed. I could be overestimating it.

I'm also not sure how much a browser app can do. A web client sits above the operating system and the browser, and some defenses simply aren't available to it. I'd like the documentation to be honest about that instead of promising more than it can deliver.

The takeaway I've settled on is narrow. Encryption covers the path between devices. Agents operate at the two ends of that path. For a small project, the useful response is to be specific about what the system protects, what it doesn't, and who controls the parts it doesn't.

Halonyx is at github.com/ABHIRAM-CREATOR06/Halonyx. Review of the threat model and the contributions are truly welcome.

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.