InfrastructureCompute, chips & cloud
NVIDIA Publishes Guide to Deploying Dynamo on Kubernetes With AICR Validated Recipes
NVIDIA's walkthrough shows how AICR recipes pin and validate a GPU Kubernetes stack before Dynamo is deployed. It compares frontend KV routing with Gateway API endpoint picking, but it targets Dynamo 1.4.2, and Dynamo 1.5.0 has since changed the default endpoint picker.
From the NVIDIA Generative AI article of October 30, 2026 and cited sources: AICR turns a snapshot of a cluster into bundles of GPU, scheduling, metrics and ingress components with fixed versions, then checks the live cluster before any model is deployed. [1] [4] [7] [8]
SOURCES[1] Build a Validated Kubernetes Runtime for NVIDIA Dynamo… developer.nvidia.com[4] AICR releases github.com[7] NVIDIA/aicr GitHub github.com
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- AICR turns a snapshot of a cluster into bundles of GPU, scheduling, metrics and ingress components with fixed versions, then checks the live cluster before any model is deployed. [1] [4] [7] [8]
- Original source
- NVIDIA Generative AI article, company source
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- When
- October 30, 2026 article published
- Covered
- Oct 31 edition
- Evidence
- 9 sources across 6 sites
- 1Build a Validated Kubernetes Runtime for NVIDIA Dynamo…developer.nvidia.com
- 2Linked video: Build a Validated Kubernetes Runtime for…youtube.com
- 3Dynamo deprecationsdocs.nvidia.com
- 4AICR releasesgithub.com
- 5KubeCon 2026 EU Day 1 Recap (Virtualization Review)virtualizationreview.com
- 6Dynamo v1.5.0 releasegithub.com
RisingHow it’s measured
YouTube video · 9K views · 70 likes
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
AICR turns a snapshot of a cluster into bundles of GPU, scheduling, metrics and ingress components with fixed versions, then checks the live cluster before any model is deployed. The guide uses the AICR v1.0.0 recipe for EKS on GB200/GB300 NVL72, which pins dynamo-platform 1.4.2, and can output Helmfile, Helm, Argo CD or Flux bundles. [1] [4] [7] [8]
The guide covers two routing paths. In the first, the Dynamo frontend router scores workers by how much of the request's prefix they already have cached and by their current load. In the second, agentgateway sends traffic through the Gateway API Inference Extension to an endpoint picker that runs Dynamo's KV router. The recipe uses live KV-cache events from workers because predicting cache state misses evictions. [1] [9]
AICR pairs a fixed set of component versions with checks on the cluster before deployment, and the recipe documents two cache-aware routing paths.
Read the full assessment
Implication: platform teams get an acceptance test before serving models, but upgrade plans must account for Dynamo 1.5's routing changes.
Sources are checked; writing the dossier now.
Executive brief
NVIDIA's walkthrough for running Dynamo on Kubernetes was published 30 October 2026, and it already trails both projects it covers. It pins Dynamo 1.4.2 under AICR v1.0.0. Dynamo 1.5.0 shipped in September 2026 with a Rust Endpoint Picker as the default, and the post treats that change as future work (Dynamo deprecations). AICR itself had reached v1.1.2 by 21 October (AICR releases). AICR turns a snapshot of a cluster into tested, version-pinned bundles of GPU, scheduling, metrics and ingress components, then checks the live cluster before any model goes on it. No independent coverage or evaluation of this specific recipe was found.
What changed and event timeline
AICR announced at KubeCon EU
NVIDIA introduced AICR as machine-readable recipes that replace hand-tuning of more than 250 configuration values. At the same event it donated its DRA GPU driver to CNCF ().
Dynamo 1.4.0 turns off bundled NATS
The NATS subchart is now off by default and must be enabled explicitly. Kubernetes-native discovery and a TCP request plane take its place ().
Dynamo 1.5.0 ships a Rust EPP
The Go Endpoint Picker was removed and a Rust EPP became the default.
eppConfigis deprecated and CRD storage moved to v1beta1 (;).AICR v0.21.0 pins Dynamo 1.4.2
This release moved dynamo-platform to 1.4.2, made NATS optional, and added GB300 and Vera Rubin support ().
AICR v1.0.0 goes stable
This release sets a v1 stability contract for the CLI, API, SDK and schemas. NVIDIA says validation now proves GPU and fabric capability instead of assuming it. v1.1.0 followed on 19 October ().
NVIDIA publishes the Dynamo-on-AICR walkthrough
The post compares two routing paths and gives a five-step deployment flow, using the AICR v1.0.0 recipe for EKS on GB200/GB300 NVL72 ().
Capabilities and access
- Versions: AICR v1.0.0 with dynamo-platform 1.4.2, which includes the operator and the
nvidia.com/v1beta1DynamoGraphDeployment CRD (NVIDIA blog). - Components pinned in the recipe: GPU Operator v26.7.1, DRA GPU Driver 0.5.0, KAI Scheduler v0.16.9, Grove v0.1.0-alpha.13, NVSentinel v1.25.0, agentgateway v1.5.0 and kube-prometheus-stack 84.4.0.
- Output formats: Helmfile, Helm, Argo CD and Flux bundles (AICR repo).
Read the full section
- Versions: AICR v1.0.0 with dynamo-platform 1.4.2, which includes the operator and the
nvidia.com/v1beta1DynamoGraphDeployment CRD (NVIDIA blog). - Components pinned in the recipe: GPU Operator v26.7.1, DRA GPU Driver 0.5.0, KAI Scheduler v0.16.9, Grove v0.1.0-alpha.13, NVSentinel v1.25.0, agentgateway v1.5.0 and kube-prometheus-stack 84.4.0.
- Output formats: Helmfile, Helm, Argo CD and Flux bundles (AICR repo).
- Open source, with SLSA Build Level 3 provenance for images (AICR docs).
Technical analysis for researchers and developers
- Event plane vs request plane: vLLM publishes KV-cache events over ZMQ. Dynamo's worker runtime normalizes them and republishes them.
- Path 1, Frontend router (
DYN_ROUTER_MODE=kv): the frontend scores workers by prefix overlap and current load. - Path 2, GAIE/EPP: agentgateway sends an HTTPRoute to an InferencePool.
Read the full section
- Event plane vs request plane: vLLM publishes KV-cache events over ZMQ. Dynamo's worker runtime normalizes them and republishes them. Requests travel separately over TCP.
- Path 1, Frontend router (
DYN_ROUTER_MODE=kv): the frontend scores workers by prefix overlap and current load. NVIDIA calls this the smallest and most production-proven topology. - Path 2, GAIE/EPP: agentgateway sends an HTTPRoute to an InferencePool. The EPP runs Dynamo's Rust KV router inside a Go framework through CGO, and a frontend sidecar on each worker in
directmode carries out the EPP's choice. - Predicted vs live cache state: the router can predict cache state instead of reading worker events, but prediction misses evictions, so its view drifts. The recipe turns live events on (NVIDIA blog).
Claims and evidence
- AICR's main claim, that tested version sets make deployments reproducible, comes from NVIDIA alone (NVIDIA blog). No third-party test of these recipes was found.
- NVIDIA says the Rust EPP reduces per-request overhead, but publishes no benchmark figure for it.
- Dynamo's own videos describe the operator pattern: the team declares the desired state and the operator reconciles the cluster to match it (01:02).
Read the full section
- AICR's main claim, that tested version sets make deployments reproducible, comes from NVIDIA alone (NVIDIA blog). No third-party test of these recipes was found.
- NVIDIA says the Rust EPP reduces per-request overhead, but publishes no benchmark figure for it.
- Dynamo's own videos describe the operator pattern: the team declares the desired state and the operator reconciles the cluster to match it (01:02). It offers two entry points: an intent-based DGDR, which profiles the setup and generates a DGD, or a hand-written DGD (01:38). The video covers the application layer only and does not mention AICR.
Context and prior work
- GAIE added InferencePool and endpoint pickers that route on metrics such as queue depth and KV-cache use (CNCF blog).
- The CNCF sandbox project llm-d covers overlapping ground: KV-cache management and separating prefill from decode (Virtualization Review).
- The video's case is that Kubernetes-native serving replaces custom glue for discovery, scaling and routing (03:25).
Read the full section
- GAIE added InferencePool and endpoint pickers that route on metrics such as queue depth and KV-cache use (CNCF blog). Dynamo plugs its own KV router into that interface.
- The CNCF sandbox project llm-d covers overlapping ground: KV-cache management and separating prefill from decode (Virtualization Review).
- The video's case is that Kubernetes-native serving replaces custom glue for discovery, scaling and routing (03:25).
Limitations, safety and contested findings
- Version drift: the guide covers Go/CGO EPP and
eppConfigbehaviour that Dynamo 1.5.0 has already removed (deprecations). - Alpha component: Grove is pinned at an alpha release.
- Narrow validation scope: checks differ by recipe, and the performance phase runs separately because it uses the GPUs under test.
Read the full section
- Version drift: the guide covers Go/CGO EPP and
eppConfigbehaviour that Dynamo 1.5.0 has already removed (deprecations). - Alpha component: Grove is pinned at an alpha release.
- Narrow validation scope: checks differ by recipe, and the performance phase runs separately because it uses the GPUs under test.
- Ingress exposure: agentgateway exposure depends on the user setting
allowedSourceRangescorrectly (NVIDIA blog).
Business and practitioner implications
- Platform teams can treat AICR as a bill of materials with an acceptance test attached: run validation before any DynamoGraphDeployment goes on the cluster.
- Pick the routing path by who owns ingress.
- Upgrade plans should allow for the 1.5 changes: the Rust EPP and dropping
eppConfigforDYN_*environment variables.
Read the full section
- Platform teams can treat AICR as a bill of materials with an acceptance test attached: run validation before any DynamoGraphDeployment goes on the cluster.
- Pick the routing path by who owns ingress. Use the Frontend router for endpoints dedicated to one model, and GAIE/EPP when a shared Gateway API layer should handle retries and fallback.
- Upgrade plans should allow for the 1.5 changes: the Rust EPP and dropping
eppConfigforDYN_*environment variables. - Agentic workloads with long, repeated prefixes stand to gain the most from cache-aware routing.
The source trail.
Sources (9)
Build a Validated Kubernetes Runtime for NVIDIA Dynamo (NVIDIA)
developer.nvidia.comLinked video: Build a Validated Kubernetes Runtime for NVIDIA Dynamo
linked video manual captions
www.youtube.com