NVIDIA AICR Guide Pins Dynamo Deployment to Versions and Cluster Setup

NVIDIA's AI Cluster Runtime (AICR) gives Kubernetes operators a recipe-based way to configure Dynamo, the company's distributed inference framework, on an existing cluster and validate the deployment against a defined target.

AICR divides the process into Snapshot, Recipe, Bundle and Validation. Snapshot captures cluster state; Recipe resolves a target configuration; Bundle renders deployment files; Validation checks the running cluster and deployed components against the recipe. Operators can install rendered files with tools such as Helm or Helmfile. AICR applies its configuration workflow to a Kubernetes cluster that is already running.

An Amazon Elastic Kubernetes Service (EKS) example uses NVIDIA H100 GPUs and Ubuntu nodes. It pairs AICR v1.1.2 with Dynamo Platform Helm chart 1.4.2 in the dynamo-system namespace. Dynamo recipes use Dynamic Resource Allocation (DRA), which requires Kubernetes 1.34 or later.

Dynamo has two request-routing paths. Its Frontend router selects workers using KV-aware scoring. The Gateway API path uses the Gateway API Inference Extension (GAIE): the gateway calls the Dynamo Endpoint Picker Plugin (EPP), which selects a worker, then forwards the request to that worker's Frontend sidecar. The installation details and routing choices address different parts of deployment: the EKS example defines an environment, while the routing paths explain how requests reach a worker.

Dynamo Platform 1.4.2 defaults its request plane to TCP and KV events to ZMQ. Bundled NATS remains optional and is not installed by default.

AICR's catalog also lists EKS inference recipes for GB200 and GB300 accelerators. A pass status means a recipe resolves and its charts are pinned; it does not show that the hardware was validated. Both catalog rows list R:0, D:4, P:1 and C:11 for readiness, deployment, performance and conformance coverage. Performance validation is a separate phase, so the catalog entries do not establish an inference speed or cost result.