KubeDay Japan 2024

Kubernetes encapsulates the details of infrastructure so that we can describe our desired state as reusable manifest. However, such nature prevents us from maximizing utilization of peripheral resources because hardware topology heavily impacts on the performance of computation workloads. For example, inter-device and inter-node communication acceleration technologies like Direct Memory Access (DMA) become unavailable if multiple GPUs under different PCI Express Bridge are allocated to a single set of AI workloads. This presentation shows a concrete usecase of Kubernetes Scheduling Framework plugins and Device Plugin to solve the issue. They work in harmony to allocate preferred AI hardware devices to computing workloads. In addition to utilizing existing stable technologies, the mechanism with Dynamic Resource Allocation (DRA), a promising technology for device management in Kubernetes, will also be included.