Why AWS Had AI Agents Build an Offline Cloud Emulator

When you set an autonomous coding agent loose on a cloud-native codebase, it runs into an immediate wall: it cannot run the code.

Unlike pure JavaScript or Python scripts that run in a local subprocess, cloud workflows depend on services like SQS, DynamoDB, and Lambda. If an agent cannot talk to those services, it guesses. It hallucinates parameters, assumes synchronous behavior for asynchronous operations, and invents configuration flags that sound reasonable to a language model but fail in production.

Pointing the agent at a live AWS account brings its own headaches: high network latency on every iteration, surprise billing spikes from agent loops running overnight, and real security risks if credentials leak into agent context.

On November 11, Amazon CTO Werner Vogels and Swami Sivasubramanian announced AWS Emulator, releasing a local offline emulator covering more than 200 AWS services under the CC0 public domain license on GitHub.

What makes the release notable is not just the offline environment. It is how AWS built it. Autonomous AI agents wrote every line of emulator code, generated the tests, diagnosed the bugs, and submitted every commit. According to Matthew Goldberg, Senior Principal Engineer on the AWS Developer Tools team, human engineers wrote zero lines of implementation code and reviewed zero pull requests.

Generating ground truth from Smithy models

Building an emulator for 200 cloud services by hand would take a team years. AWS automated the process because they had two ingredients that most software projects lack: a formal machine-readable specification and a live reference implementation.

The specification came from AWS's public Smithy models, which formally define every operation, input payload, output schema, and error type. The reference implementation was the actual AWS cloud.

Goldberg organized the agents into three specialized teams:

1. Test Writers: These agents read the Smithy models and synthesize end-to-end customer scenarios. For example, a test creates an SQS queue, sends a message, confirms visibility timeouts, receives the payload, and deletes the queue. The agents execute each test against real AWS to verify that it reflects genuine cloud behavior. Once verified, the scenario is saved to an internal repository as a deterministic fidelity test. Tens of thousands of these tests now live in the open source aws-emulator-evaluation-suite, runnable locally with no network access.

2. Issue Finders: These agents replay the fidelity tests against the local emulator codebase. The project started as a blank HTTP server handling routes for 438 Smithy models while returning empty responses. Whenever a test output diverged from real AWS behavior, the agents logged an issue in issues/things-that-do-not-work and clustered related failures by probable root cause.

3. Fixers: Hundreds of agents pick up one issue each. An agent inspects the code, traces the discrepancy, patches the emulator, and submits the change to an automated merge gate.

Hard boundaries and the merge gate

Unconstrained agents will take wild engineering shortcuts. During early development, when agents needed to support AWS Lambda functions locally, they did not reach for standard container runners. Instead, they began building their own Python interpreter from scratch. The human team had to step in with an explicit constraint directing the agents to use existing system libraries and runtimes.

To maintain code quality without human reviewers, every commit had to pass an automated merge gate:

• Static analysis: Type checking, compilation, and linter enforcement.
• Scope fences: Hard path checks ensuring an agent fixing S3 could not modify IAM or DynamoDB code.
• Adversarial security review: Dedicated security agents auditing the diff for shell injection, unsafe deserialization, and privilege leaks.
• Regression testing: Replaying the entire library of recorded fidelity tests to ensure previous capabilities remained intact.

Only changes that cleared every check reached the mainline branch. Failing changes returned to the fixer agent with diagnostic logs for another attempt.

Harnesses beat prompt engineering

In his announcement note, Swami Sivasubramanian noted that twenty years ago, building the replication engine for Alf, Amazon's early Paxos-based locking service, took four months of manual distributed systems modeling. Recently, he tried rebuilding that replication engine with Amazon's Kiro agent. Without an automated test harness, he spent four nights babysitting prompt outputs that failed on subtle concurrency edge cases.

Once he gave the agent a formal spec and an execution harness that simulated network partitions, the rebuild finished in two hours.

The difference between four nights of prompt tweaking and two hours of automated code generation comes down to feedback speed. When coding agents have access to local execution environments with deterministic pass-fail signals, they converge rapidly. Without an execution loop, they drift.

For engineers adopting autonomous coding agents, the priority is shifting away from prompt phrasing toward harness design: formal interface schemas, recorded behavioral ground truth, and automated merge gates that catch drift before it enters the codebase.