Zesty announced support for NVIDIA’s CUDA-native GPU fractioning. Following NVIDIA’s open-source release of Run:ai as part of the NVIDIA DSX AI Factory Platform, Zesty’s Automated Fractional GPUs adds the autonomous layer that continuously decides GPU memory allocations and compute sharing modes, as well as steers workload placement as AI workload demand fluctuates. With these autonomous capabilities, teams can improve GPU utilization, run more workloads on existing infrastructure, reduce the need for excess GPU capacity, and lower infrastructure costs while protecting application performance.
66% of organizations hosting generative AI models run some or all of their inference on Kubernetes, and GPU capacity for those workloads is expensive and often constrained. Teams have traditionally allocated entire GPUs to individual workloads or shared them using time-slicing, NVIDIA Multi-Process Service (MPS), or NVIDIA Multi-Instance GPU (MIG), which divides each GPU into fixed hardware-based partitions. Allocations cannot keep pace with dynamic AI workloads and real-time changes in demand. As a result, GPU memory and compute end up stranded, and teams buy more GPUs even while existing capacity sits unused.
NVIDIA’s recent open-source release of Run:ai brings CUDA-native GPU fractioning to Kubernetes through the GPU Fractioning Operator, which enforces per-workload GPU memory limits and works with Kubernetes schedulers. The fractioning mechanism makes flexibility possible and defines how workloads can divide a GPU but does not determine how to allocate that capacity. Deciding which workloads share each GPU, how much memory each one receives, and how those allocations should change as demand rises and falls requires a separate decision layer.
Zesty provides that layer. Its Automated Fractional GPU capability continuously determines which workloads should share each GPU, how many can safely run together, how much GPU memory each one receives, which sharing mode fits each workload, and when those allocations should increase or decrease. Decisions draw on actual GPU memory usage and utilization, observed workload behavior over time, and the latency requirements and resource limits defined by teams beforehand. Every change stays within the performance and availability policies teams set for each workload.
The capability deploys as a lightweight agent in the Kubernetes cluster and works alongside NVIDIA Run:ai GPU Fractioning Operator and the Kubernetes scheduler, supporting both time-slicing and MPS. It requires no application code or model changes, and installation does not restart running workloads. On a supported GPU stack, Zesty manages the Kubernetes configuration needed for optimization without manual manifest changes.
Automated Fractional GPUs is available now as part of the Zesty platform.
For more information, visit zesty.co/automated-fractional-gpus.
The post Zesty adds autonomous GPU fractioning for Kubernetes AI workloads appeared first on Engineering.com.