[Initiative]: Scheduler Capabilities Whitepaper

Start with the linked tracking document and the prior scheduling series to understand the initiative's scope and methodology. The deliverable is a whitepaper evaluating vanilla Kubernetes 1.36+ and Kueue against representative AI workloads, including throughput, utilization, scheduling time, and fairness; the issue does not identify code files or tests. Done means publishing the framework and initial evaluation described in the exit criteria.

Name

Scheduler Capabilities Whitepaper

Short description

Write a way to characterize schedulers for batch workloads

Responsible group

TAG Workloads Foundation

Does the initiative belong to a subproject?

Yes

Subproject name

Batch Subproject

Primary contact

@catblade

Additional contacts

@catblade @stackedsax @faganihajizada @zhanyl-tech

Initiative description

A Kubernetes cluster rarely runs one workload at a time. It runs a mix of training jobs, inference services, and batch jobs, spanning multiple tenants and priorities, each submitted as one or more workloads. How that mix behaves depends on scheduler configuration: whether a workload schedules together or stalls partway, how long each workload waits its turn, how often it is preempted, and how fairly capacity is split across tenants over time. Documentation describes each capability on its own. It does not describe what happens when several capabilities act on the same mix of competing workloads at once. Fair-share allocation, backfill, gang scheduling, preemption, and topology-aware placement can each behave as documented in isolation and still produce conflicts, starvation, or degraded priority handling once combined. Kubernetes itself is formalizing more of this territory as workload-aware scheduling.
This paper provides a framework for testing these interactions against real workload behavior. It defines a workload-driven evaluation methodology that measures throughput, utilization, time to schedule for high-priority workloads, and fairness of capacity shared across users over time. It applies that methodology to fair-share allocation, backfill, gang scheduling, workload-aware preemption, and topology-aware placement, using a reference workload set representative of common AI infrastructure patterns. The results show how each capability affects workload behavior on its own, and where combinations of capabilities produce conflicts that no single capability's documentation surfaces..
The framework is scheduler-agnostic and built to be extended over time. Teams can apply the same framework to their own configurations and workload mixes before committing to a production deployment.

Deliverable(s) or exit criteria

Practitioners running AI workloads on Kubernetes have limited options when it comes to evaluating scheduler configurations. Feature documentation describes what a scheduler supports, but it does not describe how that support holds under the workload conditions a specific team will encounter in production. The gap between documented capability and observed behavior under real conditions is where most configuration decisions fall short.
This paper provides a framework for closing that gap. It defines a structured methodology for evaluating scheduling capabilities against representative AI workloads, so that teams can base configuration decisions on measured outcomes rather than feature comparisons. The framework covers what to measure, how to measure it, and how to interpret results in the context of a specific workload mix and cluster environment.
Not every environment requires every capability. A team running a small, homogeneous cluster has fundamentally different scheduling requirements than a team managing a large multi-tenant GPU cluster. The framework is designed with that in mind: it helps teams identify which capabilities matter for their workloads and evaluate those specifically, rather than producing a universal ranking of schedulers.
The initial evaluation in this paper covers two configurations: vanilla Kubernetes at version 1.36 and above, which serves as the control baseline, and Kueue, which is the first scheduler evaluated against the full framework. The baseline establishes what scheduling behavior looks like without additional tooling, providing a reference point for all subsequent comparisons. Other schedulers are invited to contribute their own evaluations using the same methodology and submit them for inclusion via the process described in the Guidance for Practitioners section. The goal is a framework the community can build on, not a fixed comparison between a predefined set of tools.
This paper is intended for infrastructure engineers, platform teams, and technical architects responsible for operating AI workloads on Kubernetes. Familiarity with Kubernetes fundamentals and the scheduling concepts covered in the prior series is assumed, but no prior knowledge of any specific scheduler is required.
The following are out of scope: hardware benchmarking, single-node compute throughput, model-level performance, and non-GPU hardware configurations including TPUs. Topology-aware placement is evaluated as a scheduling capability, but the underlying hardware and network infrastructure required to support it is treated as an environmental prerequisite rather than a subject of evaluation.

Tracking document for meeting and progress

https://docs.google.com/document/d/1GuZGyBkRGG0lEeiPA8q0PfvFlwUlwa5k-ZfXafCTdBY/edit?usp=sharing