NVIDIA Workload Map Gives Kubernetes Tools a Shared AI Format

Framework-specific Kubernetes resources store workload structure and status differently, so platform tools may need separate logic for each one. NVIDIA’s open-source Workload Map uses a common definition to help those tools inspect and handle AI workloads.

Workload Map pairs a Kubernetes custom resource definition (CRD) with a Go library. Each workload definition identifies where a framework-specific resource keeps its pod templates, replica counts and status information. The CRD resource kind is Karta.

The catalog contains 13 prebuilt workload definitions: JobSet, PyTorchJob, RayCluster, RayJob, RayService, KServe, Knative, MPIJob, NVIDIA NIM, LeaderWorkerSet, Milvus, Grove PodCliqueSet and NVIDIA Dynamo. Developers can describe additional workload types in YAML.

A JobSet example includes one leader and two worker jobs, each requesting four GPUs, for a total request of 12 GPUs. Workload Map’s command-line tool can inspect the definition and show the workload’s roles and GPU request. KAI Scheduler can use the same definition for gang scheduling and as a fallback for workload types without a native plugin.

NVIDIA Run:ai can register a workload type from the catalog and manage a JobSet as one workload. The definition can therefore be used across workload inspection, scheduling and platform management.

NVIDIA included Workload Map among four components released as open source with the Run:ai platform on Oct. 13. The project is licensed under Apache 2.0.