DRA Integration

When the DRAWorkloadResourceClaims feature gate is enabled, you can associate Dynamic Resource Allocation (DRA) ResourceClaims with a PodGroup. This creates a single ResourceClaim that is shared across all pods in the gang i.e. the claim is reserved for the PodGroup rather than for individual pods.

Prerequisites

In addition to the general WAS prerequisites, you must:

Enable an additional feature gate. Add DRAWorkloadResourceClaims to your cluster configuration:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
  GenericWorkload: true
  GangScheduling: true
  TopologyAwareWorkloadScheduling: true
  WorkloadAwarePreemption: true
  DRAWorkloadResourceClaims: true   # Enables PodGroup-level ResourceClaims
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    apiServer:
      extraArgs:
        runtime-config: "api/alpha=true"
- role: worker
- role: worker
- role: worker
- role: worker

kind create cluster --image=kindest/node:latest --config kind.yaml

How It Works

Without DRA PodGroup integration, each pod creates its own ResourceClaim. With DRAWorkloadResourceClaims enabled, you can declare resourceClaims on both the Workload’s podGroupTemplates and the PodGroup’s spec. When a pod’s resourceClaims entry matches a PodGroup’s entry (same name and same template), the scheduler reserves the resulting ResourceClaim for the PodGroup instead of the individual pod.

This means:

• One claim, many pods: A single ResourceClaim is created from the template and shared across all pods in the gang.
• PodGroup-level reservation: The claim’s status.reservedFor references the PodGroup, not individual pods — avoiding the 256-entry reservation limit.
• Lifecycle management: Claims created from templates are owned by the PodGroup and cleaned up when the PodGroup is deleted.

Step 1: Create the ResourceClaimTemplate

The ResourceClaimTemplate defines the device request. This example requests a single GPU from the gpu.example.com DeviceClass:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-claim-template
  namespace: default
spec:
  spec:
    devices:
      requests:
      - name: gpu
        exactly:
          deviceClassName: gpu.example.com

Step 2: Create the Workload

The Workload references the JobSet and defines a pod group template with gang scheduling and a ResourceClaim:

apiVersion: scheduling.k8s.io/v1alpha2
kind: Workload
metadata:
  name: training-workload
spec:
  controllerRef:
    apiGroup: jobset.x-k8s.io
    kind: JobSet
    name: training-job
  podGroupTemplates:
  - name: workers
    schedulingPolicy:
      gang:
        minCount: 6
    resourceClaims:
    - name: shared-gpu
      resourceClaimTemplateName: gpu-claim-template

The key addition compared to basic gang scheduling is resourceClaims in the podGroupTemplates entry. This declares that pods in this group share a GPU claim created from gpu-claim-template.

Step 3: Create the PodGroup

The PodGroup references the Workload’s pod group template and carries the same resourceClaims:

apiVersion: scheduling.k8s.io/v1alpha2
kind: PodGroup
metadata:
  name: training-workers
  namespace: default
spec:
  podGroupTemplateRef:
    workload:
      workloadName: training-workload
      podGroupTemplateName: workers
  schedulingPolicy:
    gang:
      minCount: 6
  resourceClaims:
  - name: shared-gpu
    resourceClaimTemplateName: gpu-claim-template

When the PodGroup is created, Kubernetes generates a ResourceClaim from the template. The claim name appears in status.resourceClaimStatuses.

Step 4: Create the JobSet

The JobSet creates 6 pods (3 replicas × 2 completions). Each pod references the PodGroup through schedulingGroup.podGroupName and declares the same resourceClaims entry:

apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
  name: training-job
spec:
  replicatedJobs:
  - name: workers
    replicas: 3
    template:
      spec:
        completions: 2
        parallelism: 2
        backoffLimit: 0
        template:
          spec:
            terminationGracePeriodSeconds: 0
            schedulingGroup:
              podGroupName: training-workers
            resourceClaims:
            - name: shared-gpu
              resourceClaimTemplateName: gpu-claim-template
            containers:
            - name: worker
              image: busybox
              command: ["/bin/sh", "-c", "sleep infinity"]
              resources:
                claims:
                - name: shared-gpu

Because the pod’s resourceClaims entry matches the PodGroup’s entry, this triggers PodGroup-level reservation as described above.

Verify

Check that the PodGroup is scheduled and the ResourceClaim is allocated:

kubectl get podgroup training-workers
kubectl get resourceclaim

The ResourceClaim should show allocated,reserved. Verify that the reservation targets the PodGroup:

kubectl get resourceclaim -o jsonpath='{.items[0].status.reservedFor[0]}'

The output should reference podgroups rather than pods:

{"apiGroup":"scheduling.k8s.io","name":"training-workers","resource":"podgroups","uid":"..."}

Check that all 6 pods are running:

kubectl get pods -l jobset.sigs.k8s.io/jobset-name=training-job