Dev.to AI 🤖 Ai 👁 0 📖 6 min read

What an audit log for AI coding agents should record

Originally published on sandscale.dev. Most teams think about an agent audit log the first time someone asks "what did the agent actually do?" and nobody can answer. By then the sandbox is gone and the only record is a

Originally published on sandscale.dev.

Most teams think about an agent audit log the first time someone asks "what did the agent actually do?" and nobody can answer. By then the sandbox is gone and the only record is a chat transcript that shows what the model said, not what ran.

This is a checklist for the log you want before that day, written for the person doing the incident review. Each section ends with how Sandscale's audit log covers the point, using only what our docs state, and where it does not.

Start with the questions the log has to answer

An incident review for an agent run usually comes down to five questions:

  1. Who or what started this, and on whose authority?
  2. What rules were in force at the time?
  3. What did it run, change and try to reach?
  4. Which sensitive things did it touch: secrets, data, approvals?
  5. Can I trust that this record is complete and unedited?

Every field below answers one of them. Anything that answers none of them is noise.

1. Who started the run

Record the human who started the run, the identity the agent acted as, and the client it came through: the console, an API key, a coding agent over MCP, or a scheduled job. These are often different things, and a log that only stores an API key ID sends you hunting for whoever owned that key. Keys get shared.

In Sandscale: every audit entry records who did it, when and where (audit log). Sign-ins are recorded too. When Claude or ChatGPT connects through the Sandscale connector, each call runs as the person who connected it, with a short-lived credential, and the audit entry records the tool, the region, the outcome and the app (connecting Claude and ChatGPT). SSH access uses a certificate issued to you that expires in minutes, and that is recorded as well (SSH).

2. The policy in force

When an agent deleted something, the next question is "was it allowed to?" That needs the policy in force at that moment, not today's. Record policy changes as events with who changed what, and record each decision on an agent action: allowed, blocked, or sent to a person.

In Sandscale: policy changes and use of the kill switch are written to the audit log. Claude Code runs under your policy through its hooks, so each prompt is checked and each tool call is allowed, blocked or sent to a person, and every decision is in the session record and the audit log (agent policy). Codex and Gemini CLI report every step over OpenTelemetry, but our pages describe enforcement through hooks for Claude Code only, so for those agents you get a record, not a gate.

3. Commands and file writes

For each command, record the command line, the exit code, how long it took, and whether it timed out or was killed. Output can be large and can contain secrets, so keep it in a separate store with shorter retention.

File writes are how an agent changes code and how it plants things for later, such as a modified CI config. At minimum record files moved in and out; ideally, writes to sensitive paths inside.

In Sandscale: commands run and files moved are recorded in the audit log. Every run through the API is also recorded with its details: GET /api/v1/sandbox/runs/{id} returns them, and the run's answer includes exit_code, stdout, stderr, duration_ms and the timed_out and oom_killed flags (sandboxes API guide). What our docs do not describe is a record of every individual file write a process makes inside the microVM. If you need that, capture it inside the box, for example with a diff of the working tree at the end of the run.

4. Network egress attempts and denials

Allowed connections tell you where data could have gone. Denied ones are often more telling: an agent reaching for an unknown host right after a dependency's install script ran is an early sign of a supply chain problem. Record host, port and decision for both.

In Sandscale: egress is controlled with an allowlist you set (security). Our docs do not say that individual connection attempts or denials are written to the audit log, so we will not claim it. If egress denials matter to your review process, log them yourself at the boundary you control, or ask us about it before relying on it.

5. Approvals

Record the request, who decided, when, and why, and that the approved action ran once. An approval that can be replayed is a standing permission.

In Sandscale: a named approver approves or denies with a reason, nobody approves their own request, an approved call runs exactly once, and both the request and the decision are recorded (approvals, approvals and policies).

6. Secret use

Record which secret a run used and when, never the value. A log with secret values in it is a badly protected secret store.

In Sandscale: secrets used are recorded in the audit log. When a sandbox references a Vault secret, the audit entry names the variables, not the values, and the value is never in the API's answer, a log or the run's record (give a run a secret, Vault).

7. DLP hits

When a prompt or tool call contains something that looks like a credential, record which rule matched, where, and what happened. Not the value.

In Sandscale: prompts and MCP tool calls are checked against built-in and your own patterns, a match stops the call, and what was found and where is in the audit log, never the value itself. This is on every plan (DLP).

8. Integrity

A log the person under review can edit is not evidence. Entries should be chained or signed so a change or a gap is detectable.

In Sandscale: each entry carries the hash of the one before it, so a change or a gap shows, and the chain can be verified (audit log).

Retention and export

Incidents are often found weeks later and assessments look back months, so decide retention up front, keep the audit log longer than the output logs, and make shortening it a deliberate act. Export matters because your review should not depend on a vendor's console, and assessors want a file.

In Sandscale: every plan can read its audit log in the console, and Team and Enterprise can export it for a period, with the hash chain checked for every entry in that period (evidence export, pricing). The audit log API reference documents GET /api/v1/audit-logs with filters such as action, from, to and outcome, reading and setting retention, and refusing to shorten retention while a legal hold covers the affected entries. Shortening it also needs a person and a reason. Our public pages do not list a retention period per plan, so check the retention endpoint for your organization instead of assuming a number. Sandscale does not claim a certification: the export is evidence for your own assessments, and certification stays with you and your assessor.

What to do this week

Pick a recent agent run and try to answer the five questions above from the log alone, without asking anyone. Where you get stuck is your gap list.

On Sandscale the audit log comes with every plan, including Free, so you can run that test today.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.