KubeRay Gives Kubernetes Teams a Way to Manage Ray AI Workloads

KubeRay gives teams running Ray a Kubernetes-native interface for deploying and managing AI and other distributed applications. Ray lets Python programs spread workloads across multiple machines. KubeRay is an open-source Kubernetes operator: it maps Ray workload definitions to Kubernetes resources and makes Ray’s reported status visible to Kubernetes. The arrangement lets each platform handle a separate part of the work.

Those roles begin with placement. Kubernetes decides which machines host Pods, the units used to run applications in a cluster. Ray handles work inside those Pods by distributing tasks and actors over available CPU and GPU capacity. KubeRay connects the two layers without taking scheduling duties from either one: Kubernetes manages machine-level placement, while Ray assigns application work to the resources available within each Pod.

KubeRay converts a RayCluster specification into Pods and publishes cluster status through Kubernetes status fields and events. Platform teams can use Kubernetes tools to administer node pools, set quotas, manage access controls and monitor infrastructure. Because the Ray cluster’s definition and reported state sit within the Kubernetes environment, operators can inspect it alongside the infrastructure it uses. This gives infrastructure teams one management path across cluster resources and Ray workloads.

These resource definitions are Kubernetes custom resources, which extend the object types the platform can track. Teams can describe a Ray cluster, job or serving application in Kubernetes, while KubeRay coordinates the resources needed to run them.

KubeRay’s three core resources cover different operating patterns. RayCluster represents a long-running Ray cluster. RayJob runs a job and can create an ephemeral, or temporary, cluster for the work, then clean that cluster up afterward. RayService manages applications built with Ray Serve. It also supports high-availability serving and zero-downtime upgrades. Together, the resources cover persistent clusters, batch jobs and serving applications through Kubernetes.

KubeRay also supports autoscaling and heterogeneous compute, which allows a cluster to use different resource types, including GPUs. Its tested managed Kubernetes environments include Amazon EKS, Google Kubernetes Engine and Azure Kubernetes Service. Autoscaling lets deployments adjust cluster size as compute needs change.