NIST AI RMF for AI agents: turning the framework's outcomes into evidence

The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1) was published in January 2023, before most organisations were running AI agents that call tools, write files and send messages on their own. This piece reads the framework with agents in mind: which of its outcomes an agent deployment can actually show evidence for, what that evidence looks like, and where a log file stops being enough.

What the AI RMF is, and what it is not

The framework describes itself as "voluntary, rights-preserving, non-sector-specific, and use-case agnostic". It is not a certification scheme and it does not set technical requirements. Its Core is organised into four functions, GOVERN, MAP, MEASURE and MANAGE, each split into categories and subcategories that describe outcomes. NIST is explicit that these "do not constitute a checklist, nor are they necessarily an ordered set of steps". Suggested actions for each subcategory live separately in the AI RMF Playbook.

In July 2024 NIST added a companion, the Generative AI Profile (NIST AI 600-1), which applies the same Core to generative systems. Most agents today are built on generative models, so both documents are relevant.

That framing matters for anyone writing about "AI RMF compliance". There is no such thing in a formal sense. What an organisation can do is show, for each outcome it has chosen to adopt, that the outcome is being achieved. For agents, that means producing records.

Accountable and transparent, in NIST's words

Of the trustworthiness characteristics the framework lists, the one that maps most directly onto agent records is section 3.4, Accountable and Transparent. It opens: "Trustworthy AI depends upon accountability. Accountability presupposes transparency." The scope it gives transparency includes "how and when deployment, post-deployment, or end user decisions were made and by whom", and it notes that transparency "is often necessary for actionable redress related to AI system outputs that are incorrect or otherwise lead to negative impacts".

The framework is also careful about the limits. "A transparent system is not necessarily an accurate, privacy-enhanced, secure, or fair system." But "it is difficult to determine whether an opaque system possesses such characteristics". For an agent, opacity usually does not come from the model weights. It comes from not knowing what the agent did: which tools it called, with what arguments, and what came back.

The subcategories that need agent records

Most of the Core is about organisational process. A handful of subcategories cannot be satisfied for an agent deployment without a record of what the agents did. Quoting them directly:

• GOVERN 1.6: "Mechanisms are in place to inventory AI systems". For agents, the inventory question is not just which models are licensed but which agents ran, under which configuration, with which tools enabled.
• GOVERN 3.2: "Policies and procedures are in place to define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems." Showing this works in practice means being able to tell, for a given action, whether a human approved it or the agent acted alone.
• MEASURE 2.4: "The functionality and behavior of the AI system and its components ... are monitored when in production."
• MEASURE 2.8: "Risks associated with transparency and accountability ... are examined and documented."
• MANAGE 4.1: "Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and evaluating input from users and other relevant AI actors, appeal and override, decommissioning, incident response, recovery, and change management."
• MANAGE 4.3: "Incidents and errors are communicated to relevant AI actors, including affected communities. Processes for tracking, responding to, and recovering from incidents and errors are followed and documented."

The Generative AI Profile adds detail on the incident side. It says that "documentation practices including logging, recording, and analyzing GAI incidents can facilitate smoother sharing of information with relevant AI Actors", and singles out "third-party inputs and plugins" as especially important because content delivered through them "is often distributed, with inconsistent or insufficient access control". For an agent, plugins and tools are exactly where the risk sits; our piece on MCP tool-call logging goes into what to capture.

Monitoring is not the same as evidence

Every one of those subcategories can be "satisfied" by an application log that the agent's own process writes. The framework does not say otherwise. But there is a gap between monitoring and evidence, and it shows up the first time the record is disputed.

MANAGE 4.3 asks for incidents to be communicated to affected parties. Suppose an agent deleted a customer's data, or sent an email it should not have. The customer, an auditor or an insurer asks what happened. If the only record is a log the agent process wrote, on infrastructure the operator controls, the answer is a statement by the operator about itself. That may be honest. It is not something the other party can check.

The framework hints at this in MEASURE 1.3, which asks for "internal experts who did not serve as front-line developers for the system and/or independent assessors" to be involved in assessments. Independent assessment only works if the assessor can rely on the records they are given. Three properties make that possible:

• Integrity. The record is signed, so changing it after the fact is detectable.
• Separation. The key that signs the record is not reachable by the agent being recorded. Otherwise a compromised or prompt-injected agent can write its own alibi. Last week's piece on SLSA build levels explains why this decides whether a record counts as evidence.
• Time and ordering from outside. A signature does not say when something happened. A trusted timestamp or an entry in a witnessed transparency log does.

A practical test for an AI RMF write-up: for each subcategory you claim, name the record that shows it, who wrote that record, and whether someone outside your organisation could verify it without trusting you. Where the answer to the last question is no, say so in the profile rather than leaving it implied.

Writing it into a profile

The framework expects organisations to describe their position as profiles: a Current Profile that "indicates how AI is currently being managed", and a Target Profile that "indicates the outcomes needed to achieve the desired or target AI risk management goals". The gap between them becomes the action plan. NIST "does not prescribe profile templates".

For an agent deployment, a useful way to fill in the record-related rows is to grade each one honestly:

1. No record. Agent actions are not captured, or only the final output is.
2. Self-written log. Tool calls and results are logged by the agent harness. Useful for debugging and MEASURE 2.4, weak for MANAGE 4.3 disputes.
3. Signed record, key outside the agent. Each run produces a signed record written by a process the agent cannot control.
4. Signed and externally anchored. As above, with records timestamped or logged by a third party, so ordering and existence can be checked independently.

Moving a row from 2 to 3 is usually a change in where signing happens, not a new system. Moving from 3 to 4 means submitting a hash to an outside service. Neither requires the agent's behaviour to change.

What signed records do not cover

It would be easy to overclaim here, so to be exact. A signed record of an agent run supports the transparency and incident-handling outcomes above. It does nothing for MEASURE 2.11 on fairness, MEASURE 2.10 on privacy, or the validity and reliability testing in MEASURE 2.5. And a signature attributes a claim; it does not make the claim true. Our own receipt specification says it plainly: a receipt "proves that the operator's key signed the attestation. It does NOT prove the operator is honest." The AI RMF is a risk framework, not a cryptographic one, and records are one input to it.

What to take from this

• There is no AI RMF certificate. Show outcomes, subcategory by subcategory, for the ones you adopt.
• Agent records carry GOVERN 1.6, GOVERN 3.2, MEASURE 2.4, MEASURE 2.8, MANAGE 4.1 and MANAGE 4.3. Start there.
• Separate the signer from the agent. It is the cheapest change with the largest effect on whether a record will hold up.
• Grade your records in the Current Profile. Being explicit about "self-written log" is more credible than implying more.

The Traceseal receipt format, verifier and log are open: the spec, the verifier on PyPI, and the public log.

Originally published at traceseal.io. Traceseal issues signed execution receipts for AI agents: an open spec and an open verifier, one command to check.