Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 9 min read

Find out what your AI tools actually send home

Startup connections, telemetry defaults, update channels, dependency pings: four checks you can run this afternoon. On November 16, a researcher reverse-engineering Windows 11 found a system component, MissionControlAgg

Find out what your AI tools actually send home

Startup connections, telemetry defaults, update channels, dependency pings: four checks you can run this afternoon.

On November 16, a researcher reverse-engineering Windows 11 found a system component, MissionControlAggregator.dll, that records which AI coding agents run on the machine. The feature is called CLICodingAgentUsage. It keeps a bitmap of which agent CLIs have been used (Codex, Copilot, Cursor and others), and that record travels with the "optional diagnostic data" setting, which is on by default. The researcher verified the behavior in a Windows 11 ARM64 virtual machine on build 26200 and reported seeing no prompts or code in the payload, only the usage record. Microsoft had not commented as of this writing (xusheng.dev, November 16, 2026).

Nobody found that by reading a privacy policy. Someone ran the binary and watched what it did.

The same method works for every AI tool on your machine. The privacy page tells you what the vendor intends to collect. Traffic tells you what the software does. Between those two there are four channels where data leaves without you asking: the connections a tool opens at startup, the telemetry it sends by default, the update checks it runs on its own schedule, and the reporting built into its dependencies. Each channel can be checked from outside, with tools you probably already have, in about half an hour per machine.

Start with DNS

Before a process can send anything, it has to look up a hostname. Watching DNS for a few minutes while you launch a tool gives you the destination list with almost no setup.

sudo tcpdump -i any -n port 53

Run that in one terminal, launch the AI tool in another, and read the lookups. On a quiet machine the output is short enough to read line by line. An inference tool looking up its API domain is expected. The lines worth attention are the other ones: analytics domains, error trackers, a CDN that only hosts an update manifest.

If the machine is too noisy for a live capture, move the question to the resolver. A Pi-hole on the network, or NextDNS on the device, logs every lookup per client. Let it run for a working day, then filter the log by the hours you used the tool. You get the same list, with timestamps, and you did not have to babysit a terminal.

DNS gives you who. It does not tell you what was sent or which process asked. The next two checks fill that in.

Tie the connections to processes

On macOS and Linux, lsof shows open sockets per process:

lsof -i -P -n | grep -i <toolname>

Run it while the tool is working and you see every live connection with the destination address and port. Run it while the tool is idle and you learn something subtler: which connections stay open when nothing is happening.

On Linux, strace catches connections that are too short-lived for lsof:

strace -f -e trace=network -o /tmp/net.log <tool> --version

A version check should not need the network. It often uses it anyway, because the same startup path runs an update ping. The log file shows every socket call the process and its children made.

On macOS, an outbound firewall does this continuously. LuLu (free, from Objective-See) or Little Snitch prompts on every new outbound connection per process. The first day is noisy. After that, the accumulated ruleset is an inventory of everything on the machine that talks to the network, which is exactly the document you were trying to produce.

Put a proxy in the middle

Hostnames and processes still leave one question open: what is in the payload. For HTTP traffic, mitmproxy answers it.

mitmproxy
HTTPS_PROXY=http://127.0.0.1:8080 HTTP_PROXY=http://127.0.0.1:8080 <tool>

Most CLIs honor the proxy environment variables. After you trust mitmproxy's certificate (its docs cover this per platform), you see full request paths, headers, and body sizes. A telemetry POST with a 2 KB JSON body is a different finding from one shipping 400 KB per request.

Some tools pin their certificates and refuse to connect through the proxy. That refusal is also a finding: the tool insists on a channel you cannot inspect. Whether that is acceptable depends on what data the tool can reach, and it belongs in your notes either way.

If the machine is Windows

The record that opened this piece was found on Windows, and the checks above have direct equivalents there. In an elevated prompt, netstat -b -n lists connections together with the executable that opened them. pktmon has shipped with Windows since 2018 and captures packets without installing anything: pktmon start --capture, reproduce the behavior, pktmon stop, then convert the trace to a format Wireshark reads with pktmon's conversion command. For per-process network events with filters, Sysinternals Process Monitor does what strace does on Linux.

The diagnostic-data channel itself has a built-in viewer. Settings, then Privacy & security, then Diagnostics & feedback, then "View diagnostic data" enables the Diagnostic Data Viewer, which shows the events Windows has queued for upload. That is the queue the coding-agent usage record rides in, so you can read for yourself what your machine is about to say about you.

where one workstation's AI-tool traffic goes
Example scenario: DNS lookups from one developer workstation over 24 hours, grouped by destination category. Inference calls dominate by count, and nobody is surprised by them. The telemetry category is the one that was never discussed with anyone.

Telemetry is a setting, and the default is usually on

Almost none of what the capture finds is hidden. It is documented, toggled by an environment variable or a settings key, and enabled by default. A partial list that covers a lot of developer machines:

export DO_NOT_TRACK=1                # consoledonottrack.com convention
export DOTNET_CLI_TELEMETRY_OPTOUT=1 # .NET CLI
export HOMEBREW_NO_ANALYTICS=1       # Homebrew
export NEXT_TELEMETRY_DISABLED=1     # Next.js
export HF_HUB_DISABLE_TELEMETRY=1    # Hugging Face libraries
export SAM_CLI_TELEMETRY=0           # AWS SAM CLI

Azure CLI keeps its switch in config rather than the environment (az config set core.collect_telemetry=false), and the Google Cloud CLI does the same (gcloud config set disable_usage_reporting true). VS Code reads "telemetry.telemetryLevel": "off" from its settings. On Windows, the same Diagnostics & feedback page from the section above holds the off switch: turning off optional diagnostic data closes the upload queue you just read.

Each project documents its toggle, usually with a reasonable explanation of what gets collected. The pattern across all of them is the direction of the default. Collection is on until the user finds the page that says how to turn it off. The party that sets the default is the party that wants the data, so the default favors collection. Your audit does not need to argue with any of this. It needs to record which toggles exist on your machine and which state they are in.

Updates and dependencies report separately

The tool you audited is not the only sender in its own install. Auto-updaters contact vendor servers on their own schedule, whether or not you use the tool that day. Model runtimes contact a registry when they pull weights. Some packages ping an analytics service at install time (many of these honor DO_NOT_TRACK). Error-reporting SDKs ship stack traces, and stack traces can carry file paths, hostnames, and fragments of whatever was in scope when the exception fired.

A data processing agreement, when you have one, covers the product's intended flows. The telemetry of the product's dependencies is typically nobody's contract. If the tool runs where confidential work happens, the dependency layer deserves the same capture treatment as the tool itself: same tcpdump, same proxy, pointed at an install and an update instead of a chat session.

The instinct generalizes past telemetry. A post on the front page of Hacker News this week describes running coding agents without handing them credentials at all (blog.antonio.cx, November 17, 2026). Separating what a tool can read from what it can transmit is one control. Deciding what it can read in the first place is the same control, applied a layer earlier.

What to write down

The capture session produces a pile of hostnames. The deliverable is smaller. For each AI tool on the machine, four lines:

  1. Endpoints observed (from DNS, lsof, or the proxy)

  2. Which of those the vendor documents, and where

  3. Which telemetry toggles exist, and their current state

  4. What you could not inspect (pinned channels, closed-source updaters)
    Keep the file in the repo next to the tooling docs and re-run the capture after major version updates, because defaults change between versions. One more line worth adding per tool: does the vendor state where inference happens, as a specific commitment, separate from where data is stored? If the documentation never says, write "unspecified." For anything processing client work, an unspecified inference location is an answer, and it is the answer you needed the audit to surface.

How we handle this at KarmaBox

KarmaBox is our product, and it exists because we wanted the answer to this audit to be short.

It is a device you run yourself (one GB10, 128 GB of unified memory, around 170 W, sits in an office). Avatars, the AI workers on the platform, run their inference locally for local-model tasks, and business data (documents, conversations, knowledge bases, task records) stays on the device. The audit above collapses into one question: what leaves this box?

The device answers that question continuously instead of once. The data boundary panel shows outbound events as they happen. In our internal test from October 28 to 30, 2026, the device ran 197 local inference calls over 72 hours with zero business-data exits; 886 conversation segments, 2 knowledge bases, and 24 memory records stayed on the device, and the panel read "nothing has left" throughout.

Two behaviors matter for the failure cases above. Local inference fails closed: if the local model errors out, the task reports failure rather than silently rerouting to a cloud endpoint (deployed October 22, 2026). A silent fallback is precisely the kind of traffic a packet capture exists to catch, so the platform refuses to generate it. And when a task genuinely needs a cloud model, the cloud models are built into the platform, there are no API keys for you to configure, and if the call will cost money the avatar asks you first and waits. A boundary crossing becomes a decision you approved with a receipt attached, rather than a default you discover in a DNS log.

The rules sit at the platform layer, outside the model, so swapping the underlying model does not loosen them. Sensitive work goes to avatars configured for local models; public-facing work can go to cloud avatars. That split is a configuration you choose on the device, per avatar.

What this does not cover: integrations you connect (a calendar API is a new data flow with its own jurisdiction), people with physical access to the device, and the outbound channels an agent can be talked into using, which is its own problem with its own controls (AI agent data leakage covers those). And a box you run is hardware you must patch: the OS-level checks above apply to our device the same as to any other machine. If you are comparing vendors on these points, the AI vendor sovereignty checklist asks the questions in a form you can send to a sales team.

Which tool on your machine would you least like to see a day of DNS logs for?

Sources

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.