Installation

Installing Jobset to a Kubernetes Cluster

Before you begin

Make sure the following conditions are met:

  • A Kubernetes cluster with version 1.21 or newer is running. Learn how to install the Kubernetes tools.
  • Your cluster has at least 1 node with 2+ CPUs and 512+ MB of memory available for the JobSet controller manager Deployment to run on. NOTE: On some cloud providers, the default node machine type will not have sufficient resources to run the JobSet controller manager and all the required kube-system pods, so you’ll need to use a larger machine type for your nodes.
  • The SuspendJob feature gate is enabled. In Kubernetes 1.22 or newer, the feature gate is enabled by default and reached stable in Kubernetes 1.24.
  • The kubectl command-line tool has communication with your cluster.

Install a released version

To install a released version of Jobset in your cluster, run the following command:

VERSION=v0.5.0
kubectl apply --server-side -f https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/manifests.yaml

Optional: Add metrics scraping for prometheus-operator

If you are using prometheus-operator to scrape metrics from jobset components, run the following command:

VERSION=v0.5.0
kubectl apply -f https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/prometheus.yaml

If you are using kube-prometheus, metrics can be scraped without performing this step.

Customize Your Installation

You can customize the installation according to your requirements using kustomize.

For instance, if you need to modify the resource allocations for a specific deployment, follow these steps:

Step 1: Set Up Your Kustomization Environment

Start by creating a directory for your Kustomize configuration and navigate into it:

mkdir kustomize-jobset
cd kustomize-jobset

Step 2: Download the Remote Manifest

Retrieve the remote manifest file specifying the version you need:

VERSION=v0.5.0
curl -LO https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/manifests.yaml

Step 3: Create a kustomization.yaml File

Create a kustomization.yaml file that references the downloaded manifest:

resources:
  - manifests.yaml

Step 4: Define Your Resource Adjustments

Create a resource_patch.yaml file to specify your desired resource adjustments for the deployment. For example, to update the jobset-controller-manager:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jobset-controller-manager
  namespace: jobset-system
spec:
  template:
    spec:
      containers:
      - name: manager
        resources:
          requests:
            cpu: "1"      
            memory: "256Mi"  
          limits:
            cpu: "4"      
            memory: "1Gi"

Step 5: Include the Patch in Your Kustomization

Add the resource_patch.yaml file to your kustomization.yaml to apply the patch:

resources:
  - manifests.yaml
patches:
  - path: resource_patch.yaml

Step 5: Include the Patch in Your Kustomization

Apply the configuration to your Kubernetes cluster using Kustomize and kubectl:

kubectl apply -k .

Uninstall

To uninstall a released version of JobSet from your cluster, run the following command:

VERSION=v0.5.0
kubectl delete -f https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/manifests.yaml

Install the latest development version

To install the latest development version of Jobset in your cluster, run the following command:

kubectl apply --server-side -k github.com/kubernetes-sigs/jobset/config/default?ref=main

The controller runs in the jobset-system namespace.

Optional: Add metrics scraping for prometheus-operator

If you are using prometheus-operator to scrape metrics from jobset components, you need to run the following command so it can access the metrics:

kubectl apply --server-side -k github.com/kubernetes-sigs/jobset/config/prometheus?ref=main

If you are using kube-prometheus, metrics can be scraped without performing this step.

Uninstall

To uninstall JobSet, run the following command:

kubectl delete -k github.com/kubernetes-sigs/jobset/config/default

Build and install from source

To build Jobset from source and install Jobset in your cluster, run the following commands:

git clone https://github.com/kubernetes-sigs/jobset.git
cd jobset
IMAGE_REGISTRY=<registry>/<project> make image-push deploy

Optional: Add metrics scraping for prometheus-operator

If you are using prometheus-operator to scrape metrics from jobset components, you need to run the following command so it has access to the metrics:

make prometheus

If you are using kube-prometheus, metrics can be scraped without performing this step.

Uninstall

To uninstall JobSet, run the following command:

make undeploy

Optional: Use cert manager instead of internal cert

JobSet webhooks use an internal certificate by default. However, if you wish to use cert-manager (which supports cert rotation), instead of internal cert, you can by performing the following steps.

First, install cert-manager on your cluster by running the following command:

VERSION=v1.11.0
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/$VERSION/cert-manager.yaml

Next, in the file jobset/config/default/kustomization.yaml replace ../components/internalcert with ../components/certmanager then uncomment all the lines beginning with [CERTMANAGER].

Finally, apply these configurations to your cluster with kubectl apply --server-side -k config/default.

Last modified May 5, 2024: Removed comments from manifest (b0de713)