KubeDay Japan 2024

The existing countable interface for GPU resource requests often falls short in meeting the dynamic demands of compute-intensive MLOps workloads, especially LLM inferencing on Kubernetes. Traditional GPU resource management poses challenges in controlled sharing, supporting multiple GPU models, and specifying custom constraints. DRA emerges as the solution, introducing features like controlled GPU sharing within and across pods, supporting diverse GPU models on a single node, and enabling arbitrary constraints for GPUs. This not only optimizes resource utilization but also addresses the dynamic needs of modern ML workload deployment on Kubernetes. This talk specifically dives deep into how to leverage GPU resource drivers for DRA for dynamic allocation and scheduling of GPUs to manage ML workloads by showing examples of ML workload deployment on Kubernetes.