Workload Aware Scheduling

JobSet can integrate with the Kubernetes Workload Aware Scheduling (WAS) APIs (scheduling.k8s.io/v1alpha2) to enable gang scheduling and coordinated pod placement. This guide shows how to create the WAS resources alongside your JobSet.

Prerequisites

The Workload Aware Scheduling APIs require a Kubernetes 1.36+ cluster with the following alpha feature gates enabled:

• GenericWorkload
• GangScheduling
• TopologyAwareWorkloadScheduling
• WorkloadAwarePreemption

The API server must also have alpha APIs enabled via --runtime-config=api/alpha=true.

You can use Kind to create a local cluster with these feature gates enabled:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
  GenericWorkload: true
  GangScheduling: true
  TopologyAwareWorkloadScheduling: true
  WorkloadAwarePreemption: true
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    apiServer:
      extraArgs:
        runtime-config: "api/alpha=true"
- role: worker
- role: worker
- role: worker
- role: worker

kind create cluster --image=kindest/node:latest --config kind.yaml

Overview

The Kubernetes WAS APIs introduce two resources that work with JobSet:

• Workload: Represents a higher-level scheduling unit that references the JobSet and defines pod group templates with scheduling policies.
• PodGroup: Groups pods that should be scheduled together according to a shared scheduling policy (e.g., gang scheduling).

Pods in the JobSet are associated with a PodGroup through the schedulingGroup.podGroupName field in the pod spec.

Topics

• Gang Scheduling: Configure gang scheduling so that all pods in a JobSet must be schedulable before any are admitted.
• Preemption: Enable workload-aware preemption to preempt entire pod groups rather than individual pods.
• Topology Aware Scheduling: Co-locate all pods in a gang within the same network topology domain for low-latency communication.