One Line in a Pod Spec, a VM Underneath: A Practical Guide to AWS Nitro Sandboxes on EKS

AWS launched Nitro Sandboxes on November 9, 2026. The short version: add runtimeClassName: isolated-sandbox to a pod, and that pod runs in its own lightweight virtual machine, isolated by the same Nitro hypervisor that separates EC2 instances from each other.

This post is what I would want to read as a platform engineer deciding whether to adopt it: the problem it addresses, how it works, how to roll it out, what changes for your workloads, and what it does not solve.

Disclosure: I work at AWS. This post is my own and is based on the public launch blog, the Amazon EKS user guide and the public blueprints repository.

TL;DR

• Pods on a node normally share one Linux kernel. A container escape or kernel bug in one pod can reach its neighbors. Nitro Sandboxes give each opted-in pod its own kernel, behind a hardware boundary.
• Adoption is one field: runtimeClassName: isolated-sandbox (node trusted) or confidential-sandbox (node untrusted, secrets gated by attestation). Images, Helm charts, kubectl exec and logs stay the same.
• Requirements: Kubernetes 1.36 or later, a supported Nitro instance type, and EKS Auto Mode (built in) or a custom AMI.
• Some pod features behave differently: no hostNetwork, no user namespaces, no shared EBS volume, no in-place resize, no GPU passthrough at launch, and per-pod DNS and connection limits.
• Measured on a real cluster: a sandboxed pod was Ready in a median of 3.28 s against 1.38 s for an ordinary pod on the same node, and a warm-pool claim in 1.53 s. The sandboxed pod ran its own kernel (6.18.54) on a node running 6.18.56.
• It is an isolation boundary, not a policy engine. A default sandboxed pod still reached the internet in my test. You still need least-privilege identity, egress rules and, for agents, tool-level policy.

The problem: one kernel, many tenants

Containers are fast because they share the host kernel. That same property is the risk. Every syscall a container makes lands in the kernel that also serves every other pod on the node, so isolation depends on namespaces, cgroups, seccomp and LSM profiles being configured correctly and on the kernel and runtime having no exploitable bug.

The record shows that assumption breaks regularly. runc had host-escape advisories in 2019 (CVE-2019-5736) and 2024 (CVE-2024-21626), and three more on a single day in November 2025. Kernel bugs such as Dirty Pipe (CVE-2022-0847) needed no special privileges at all. A recent analysis by NoFire collects these and makes the same point: where the untrusted code runs is the question that matters.

Four trends make the shared kernel harder to accept, and the launch post names each of them:

1. AI agents run code nobody reviewed. A coding agent writes code, runs it and talks to the internet. A single prompt injection can turn that into data exfiltration or lateral movement.
2. SaaS platforms pack many customers onto shared clusters. A dedicated EC2 instance per customer is the safe answer and the expensive one.
3. Regulated industries want proof. Auditors increasingly ask for cryptographic evidence that a workload ran as expected, not a configuration checklist.
4. CI/CD jobs run privileged. docker build and similar jobs need elevated rights, so teams isolate them on dedicated node groups.

The usual workaround for all four is the same: dedicate machines and pay for idle capacity. Nitro Sandboxes aim to remove that trade-off.

What a Nitro Sandbox is

Each opted-in pod gets its own lightweight VM with its own Linux kernel. On the node, a shim (containerd-shim-nitro-sandbox-v2) starts a Firecracker microVM for the pod. Inside the VM, an agent runs the containers with its own containerd. The Nitro hypervisor sits beneath both the node and the sandbox and enforces CPU and memory isolation between them. Per the EKS user guide, each sandbox is a first-level guest with hardware-assisted virtualization, not nested virtualization or emulation.

Every sandboxed pod also gets a Cedar policy that controls which images may load, which containers may run, and whether exec and log streaming are allowed. The policy is enforced from inside the guest.

Two modes

A third switch removes the network entirely. The annotation sandbox.amazonaws.com/network-isolated: "true" boots the VM with no network device at all: no DNS, no IMDS, no Pod Identity. The user guide notes that the vsock channel still exists, so it calls this strong but not absolute isolation.

Where it fits among the options you already know

This table compares the isolation model, not performance. I have not benchmarked these options against each other, and you should measure on your own workload before deciding.

How to adopt it, step by step

Step 0. Check prerequisites

• Kubernetes 1.36 or later (eksctl users: pass --version explicitly).
• A supported Nitro instance type. Auto Mode skips FIPS AMIs, GPU and Neuron AMIs, and multi-NUMA types for sandbox nodes.
• EKS Auto Mode includes the runtime classes on every cluster. On managed or self-managed node groups, build a custom AMI with the sandbox components (guide).

Step 1. Run one isolated pod

apiVersion: v1
kind: Pod
metadata:
  name: hello-sandbox
spec:
  runtimeClassName: isolated-sandbox   # the only change
  containers:
    - name: hello
      image: public.ecr.aws/amazonlinux/amazonlinux:2023
      command: ["sh", "-c"]
      args:
        - |
          echo "product: $(cat /sys/class/dmi/id/product_name)"
          echo "disks:   $(ls /sys/block | tr '\n' ' ')"
          sleep infinity

kubectl apply -f hello-sandbox.yaml
kubectl wait pod/hello-sandbox --for=condition=Ready --timeout=300s
kubectl logs hello-sandbox

The logs are your proof. Inside a sandbox the product name ends in .enclave and the disks are virtio devices (vda, vdb), where an ordinary pod on the same node sees the instance's own NVMe disks.

Step 2. Give it identity the sandbox can use

Inside a sandbox, IMDS answers only a handful of placement paths and does not serve IAM credentials. Use EKS Pod Identity, scoped to exactly what the workload needs. In isolated mode, the service account token is not copied into the sandbox, so if a workload needs to call the Kubernetes API, mount a token explicitly (see the hardening section below).

Step 3. Close the egress

A VM boundary does not decide where traffic may go. Add egress rules. On EKS Auto Mode, an ApplicationNetworkPolicy can allow egress by hostname, for example only to Amazon Bedrock and GitHub for a coding agent. The blueprints include a side-by-side network policy sample with an ordinary pod as the control.

Step 4. Tune the per-pod limits before load testing

Two defaults surprise people under load:

• DNS: 100 queries per second per pod (max 512). Raise it with sandbox.amazonaws.com/dns-packets-per-second, or add a DNS cache sidecar.
• Outbound connections: 4,096 simultaneous per IP family. Adjust with sandbox.amazonaws.com/conntrack-limit (64 to 65,536), or pool connections.

Step 5. Move secrets-heavy workloads to confidential mode

Switch to confidential-sandbox, generate the pod's Cedar policy with eks-nitro-sandbox-genpolicy, and write KMS key policies with kms:RecipientAttestation:PCR<N> conditions. The user guide is explicit that without those KMS conditions, the attestation is recorded but never checked, so confidential mode is incomplete.

Step 6. Use warm pools for interactive sessions

Each sandbox boots a VM, so cold start is part of the design. For latency-sensitive agent sessions, the Kubernetes SIG project agent-sandbox keeps pre-booted sandboxes ready to claim. On my test cluster, claims from a five-pod warm pool were ready in a median of 1.53 seconds against 3.28 seconds for a cold sandboxed pod (details in the measurements section below). The blueprints also include a KEDA sample that scales the pool with demand.

Migration checklist: what changes for your pods

Run this against your manifests before you flip the runtime class. Source: the EKS considerations page.

Two sizing notes matter for cost:

• Density: total sandbox vCPUs on a node cannot exceed 5 times the node's vCPUs, and the sandbox count has the same cap (halved on CPU-overcommit types). Memory usually binds first.
• SMT instances: on Intel and SMT-enabled AMD types, two different sandboxes never share sibling threads of one core, so usable compute is roughly the number of physical cores. Request CPU in multiples of two, or use Graviton, which has one thread per core.

Hardening tips for sandboxed workloads

A VM boundary is the outer wall. These four habits keep the inside of the sandbox tight as well.

1. Mount only the token you need, with a short life. If a workload must call the Kubernetes API, turn off the automatic service account mount and mount a projected token with an explicit one-hour expiry. The API server can extend the automatic token well past an hour, while a projected token with expirationSeconds: 3600 stays valid for one hour and the kubelet rotates it.

spec:
  automountServiceAccountToken: false
  volumes:
    - name: kube-api-token
      projected:
        sources:
          - serviceAccountToken:
              path: token
              expirationSeconds: 3600

2. A privileged daemon can still have a read-only root. Sandboxes make it reasonable to run privileged build daemons such as dockerd or buildkitd on shared nodes. Even then, set readOnlyRootFilesystem: true and add one writable volume for each path the daemon actually needs. Add each volume only after you see the error without it, so the list stays minimal.

3. Prove a control works before you rely on it. Scanners will flag a missing seccomp profile on privileged containers. Check what actually applies with grep Seccomp /proc/1/status inside the container. On my test cluster, every privileged container showed Seccomp: 0 (no filter) even with seccompProfile: RuntimeDefault set, on an ordinary pod and on a sandboxed pod alike, while unprivileged containers showed Seccomp: 2 (filter active). Privileged mode switches seccomp off in standard Kubernetes, so for privileged build daemons the VM boundary is the control that actually holds. Document the scanner finding with that evidence rather than silencing it.

4. Use liveness probes that do not depend on what you are monitoring. If a pod's health endpoint calls the Kubernetes API or another dependency, an outage there restarts the pod and loses its state. A TCP liveness probe on the serving port avoids that.

What Nitro Sandboxes do not solve

• Tool-level policy. A VM boundary cannot express "this agent may push to git only if tests pass". Marc Brooker, one of the Firecracker authors, makes the case for pairing a policy layer with a per-session microVM in Strands Box: The Big Picture.
• A trusted node in isolated mode. In isolated-sandbox the node is trusted. If the node itself is your threat, use confidential-sandbox with attested KMS keys.
• Over-broad credentials. If the pod's IAM role can read every bucket, the sandbox does not change that. Scope identity per workload.
• GPU workloads, at launch.

What I measured on a real cluster

The docs describe the boundary. I wanted numbers, so I created a fresh EKS Auto Mode cluster (Kubernetes 1.36, us-west-2, November 12, 2026), ran ordinary and sandboxed pods side by side, and deleted the cluster afterwards. Every figure below comes from that run. Timings are client-side, from kubectl apply returning to kubectl wait --for=condition=Ready returning, so they include a little kubectl overhead that is the same for both kinds of pod.

Startup time

So a sandbox adds about 1.9 seconds to a pod start once the node is warm, and a warm pool takes almost all of that back: a claim was ready in about the time an ordinary pod takes to start. All 10 claims adopted an already-booted sandbox (launch-type: warm), and the pool was back to five ready a median of 1.4 seconds after each claim. One refill took 13.5 seconds.

The first-pod row is a single run each and depends mostly on how fast Auto Mode provisions a node, so treat it as one data point, not a benchmark.

What the pod sees

Both pods ran on the same c8a.large node with a 1 vCPU and 1,536 MiB memory request.

The kernel line is the whole story in one row: the sandboxed pod was running a different kernel from the node it was scheduled on.

Sizing facts from the node

• The isolated-sandbox RuntimeClass adds a fixed overhead of 256 MiB of memory per pod, on top of its requests.
• The 2-vCPU c8a.large node advertised 10 eks.amazonaws.com/nitro-sandbox slots, which matches the documented cap of 5 times the node's vCPUs.
• That 4 GiB node reported 2,349 MiB allocatable memory. A 2 GiB pod that would fit on many 4 GiB nodes did not fit next to one small pod here. Check allocatable on your sandbox nodes before sizing.
• For sandbox pods, Auto Mode chose c8a.large, which runs one thread per core, so the SMT note above did not apply on this run.

Egress

Each pod ran curl https://example.com with an 8-second timeout.

The middle row is the point to remember: a sandbox is not a firewall. Without the annotation or an egress policy, a sandboxed pod reaches the internet like any other pod.

What I did not measure

Throughput and latency of a real application inside the sandbox, the DNS and connection limits under load, confidential mode, and density at the cap. Those need a workload-specific test, and they are the next things I would run before production.

Where to start

1. Pick one workload that runs code you do not fully control: an agent, a CI build, or a tenant plugin.
2. Run it in isolated-sandbox with Pod Identity and an egress policy.
3. Walk the migration checklist, tune DNS and connection limits, and load test.
4. Move workloads that hold sensitive data to confidential-sandbox with attested KMS keys.

The blueprints repository has runnable samples for coding agents, warm pools, CI runners, storage and confidential workloads: aws-samples/sample-eks-nitro-sandbox. If you try it on a workload not covered there, I would like to hear what changed and what broke.

Sources: AWS Containers launch blog, Amazon EKS user guide: sandboxing and considerations, NoFire analysis. Content from these sources was paraphrased.