NVIDIA Pins the Runtime Under Dynamo to a Validated Recipe, Version by Version

TL;DR

• NVIDIA Dynamo is an open-source inference framework that coordinates model serving across GPU worker pods, starting from a DynamoGraphDeployment resource that says how vLLM, SGLang or TensorRT LLM should run.
• NVIDIA AI Cluster Runtime, AICR, packages Dynamo with the infrastructure it depends on into validated recipes, each carrying pinned component versions, compatibility constraints and validation checks.
• NVIDIA published the walkthrough on 30 October 2026, signed by Stefan Schimanski, Jamie Li and Yuan Chen, with AICR positioned inside the NVIDIA DSX AI Factory Platform.
• The recipe ships the dynamo-platform chart pinned to 1.4.2, with the Dynamo operator, the runtime images and the nvidia.com/v1beta1 DynamoGraphDeployment definition that replaces v1alpha1.
• Supporting components arrive with the recipe: GPU Operator, Node Feature Discovery, the DRA driver for NVIDIA GPUs, KAI Scheduler, Grove, NVSentinel, a Prometheus stack and agentgateway with Gateway API.
• Dynamo 1.4.2 carries requests over TCP and KV-cache events over ZMQ, the chart stops installing NATS by default, and NATS stays available as an opt-in.
• Two routing paths exist: the Dynamo Frontend router behind a Kubernetes Service with DYN_ROUTER_MODE=kv, and a Gateway API path where agentgateway resolves an HTTPRoute to an InferencePool and the Dynamo EPP selects the endpoint.
• Five steps cover the flow: aicr snapshot, aicr recipe, aicr bundle, a Helmfile apply, then aicr validate with the deployment and conformance phases before the application is installed.
• The EKS inference recipe for GB200 and GB300 NVL72 pins seventeen chart versions, Dynamo Platform 1.4.2, KAI Scheduler v0.16.9 and Agent Gateway v1.5.0 among them.

A production Dynamo deployment depends on the runtime beneath the model: GPU drivers, resource allocation, scheduling, networking and monitoring must work together. NVIDIA's technical post on 30 October 2026 addresses that lower layer for Dynamo, its distributed inference framework, and the interesting object in it is a recipe: a file that states which versions of seventeen components work together and how to prove a cluster matches them.

Capture of the NVIDIA post, developer.nvidia.com/blog/build-a-validated-kubernetes-runtime-for-nvidia-dynamo, taken 31 October 2026.

Dynamo runs on Kubernetes from a DynamoGraphDeployment, and the layer beneath decides reliability

A DynamoGraphDeployment describes the serving graph: which engine runs which stage, and how many pods it wants. The graph specifies how the serving engines should run in pods. The cluster must also provide the runtime capabilities listed in the post. Does the GPU Operator match the nodes, are GPU nodes labelled and discoverable, does the cluster expose GPUs through Dynamic Resource Allocation, can the scheduler place workers with the right topology and gang semantics, are node health signals available before a workload lands, is Prometheus reachable, and is the Gateway API path configured when ingress is Kubernetes-native.

AICR's contribution sits exactly there. It records those requirements in the recipe, and the record is what makes a cluster configuration reviewable by someone who did not build it.

The recipe pins seventeen chart versions for the EKS GB200 and GB300 build

Take the Amazon EKS inference recipe for GB200 NVL72 and GB300 NVL72 systems. AICR v1.0.0 selects seventeen Helm chart versions, and together those pins define NVIDIA’s tested compatibility set: NVIDIA GPU Operator v26.7.1, Node Feature Discovery 0.19.0, cert-manager v1.20.2, AWS EBS CSI Driver 2.59.0, AWS EFA v0.5.29, NVSentinel v1.25.0, NodeWright Operator v0.19.0, NVIDIA DRA GPU Driver 0.5.0, KAI Scheduler v0.16.9, Grove v0.1.0-alpha.13, Dynamo Platform 1.4.2, Agent Gateway v1.5.0 with its CRDs, Kube Prometheus Stack 84.4.0, Prometheus Operator CRDs 28.0.1, Prometheus Adapter 5.3.0 and Kubernetes Ephemeral Storage Metrics 1.19.2.

Read the alpha tag on Grove and the beta-shaped numbering on a few others, and the recipe's real function becomes clear: it fixes a combination that was tested, so an upgrade becomes a decision with a diff.

Netics editorial visual: what the recipe moves from memory into a file.

Requests travel on TCP and KV-cache events on ZMQ in Dynamo 1.4.2

Dynamo separates the two planes that a cache-aware router needs. Requests move over TCP by default, and KV-cache events move over ZMQ: each vLLM worker engine publishes raw cache events locally, the Dynamo worker runtime subscribes, normalises them, and republishes them on the Dynamo event plane. The post shows the vLLM flags that switch this on, --enable-prefix-caching together with a kv-transfer-config block that sets enable_kv_cache_events, a ZMQ publisher and a local endpoint.

That design also explains a quieter change. Frontends, workers and the optional EPP components discover one another through Kubernetes resources, so AICR's default path stops using NATS and the platform chart stops installing it. NATS remains an opt-in for deployments that still want it.

Two routing paths: the Dynamo Frontend router and the Gateway API with Dynamo EPP

The first path exposes a Dynamo Frontend behind a Kubernetes Service. With DYN_ROUTER_MODE=kv, the Frontend tokenises each request, scores eligible workers on prefix overlap and current load, picks one, and dispatches over the request plane. The post calls this the smallest topology and the most production-proven serving option, and it fits a team that owns one model-specific endpoint.

NVIDIA's own diagram of the first routing path: client to Service to Dynamo Frontend, then a vLLM worker selected on tokenised input and KV score, with KV events returning over ZMQ, developer.nvidia.com/blog/build-a-validated-kubernetes-runtime.

The second path puts a gateway in front. agentgateway resolves an HTTPRoute to an InferencePool, the Dynamo operator creates that pool from the DynamoGraphDeployment through a selector and an endpointPickerRef, and the Dynamo EPP tokenises the request and returns an endpoint selection to the gateway, which stays the HTTP proxy. Each routable worker pod then carries a Dynamo Frontend sidecar running with --router-mode direct, so the sidecar applies the choice without deciding again.

The split matters when a token stream slows down, because three components watch the same request from different angles: the gateway owns TLS and routing, the EPP owns the choice based on KV state, and the sidecar only forwards. Each carries its own metrics, and the validation phases check that all three are present before traffic arrives.

NVIDIA's own diagram of the second path: HTTPRoute to InferencePool to Dynamo EPP, with the selected endpoint returned to agentgateway and a worker frontend sidecar forwarding the request, developer.nvidia.com/blog/build-a-validated-kubernetes-runtime.

Five steps from snapshot to a validated runtime

The workflow keeps runtime setup apart from application deployment, which is the part worth copying even if the Dynamo specifics change.

• aicr snapshot records the cluster, defaulting to nodes labelled nvidia.com/gpu.present=true and tolerating GPU-node taints, with node selectors and tolerations available for placement.
• aicr recipe turns that snapshot plus an intent and platform into a recipe that records pinned versions, constraints and validation checks.
• aicr bundle renders the recipe into Helm values and manifests, which is where node selectors, tolerations and an agentgateway source range enter the picture.
• Deploy with helmfile diff then helmfile apply, or the generated deploy script when Helmfile is unavailable.
• aicr validate runs deployment and conformance phases against the live cluster before the DynamoGraphDeployment is installed.

Netics editorial visual: the step that turns a resolved recipe into evidence about a cluster.

Netics' take: version pinning is where an inference platform becomes operable

The failure mode this addresses is version drift across a driver, a plugin and a scheduler that previously worked together. A recipe that names seventeen versions, and a validator that compares the cluster against them, converts an argument into a diff.

The same reasoning explains why the gap between a prototype and a production service is rarely about the model. Runtime components and their versions continue to change around a deployed model.

A recipe also answers the question an auditor asks first, which is which versions produced a given result. Keep the recipe and bundle in the repository as the version record for that validation run. Netics would treat “validated” as a bounded claim: the source limits deployment and conformance checks to the selected recipe’s requirements and runs performance validation separately with a temporary GPU workload; upgrade and rollback procedures still require the operator’s own testing.

Netics editorial visual: the split between a pinned recipe and the decisions that stay with the operator.

What this means for businesses: pin, snapshot, validate, then schedule the upgrade

Netics recommends the following operating sequence for teams running inference on Kubernetes.

• Write down the versions you run, in a file, with the constraints that made them a set.
• Snapshot the cluster before a change, so the diff describes reality.
• Validate before the workload lands, and keep the report next to the change ticket.
• Put the upgrade window in a calendar owned by a named team, because a pinned recipe makes an upgrade deliberate.

Teams that run this stack on their own hardware, inside the sovereign infrastructure you operate yourself, get the same recipe with a driver stack in their own change window and a validation report they can hand to an auditor.

These controls document the runtime configuration. The recipe and validation report preserve the versions checked for that deployment.

Originally published on the Netics blog.