Custom Scheduling - openshift

Overview

You can run multiple, custom schedulers alongside the default scheduler and configure which scheduler to use for each pods.

To schedule a given pod using a specific scheduler, specify the name of the scheduler in that pod specification.

Information on how to create the scheduler binary is outside the scope of this document. For an example, see Configure Multiple Schedulers in the Kubernetes documentation.

Package the Scheduler

The general process for including a custom scheduler in your cluster involves creating an image and including that image in a deployment.

Package your scheduler binary into a container image.

Create a container image containing the scheduler binary.

For example:

FROM <source-image>
ADD <path-to-binary> /usr/local/bin/kube-scheduler

Save the file as Dockerfile, build the image, and push it to a registry.

For example:

docker build -t <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
docker push <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>

In OKD, create a deployment for the custom scheduler.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: custom-scheduler
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: custom-scheduler
subjects:
- kind: ServiceAccount
  name: custom-scheduler
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: system:kube-scheduler
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: custom-scheduler
  namespace: kube-system
  labels:
    app: custom-scheduler
spec:
  replicas: 1
  selector:
    matchLabels:
      app: custom-scheduler
  template:
    metadata:
      labels:
        app: custom-scheduler
    spec:
      serviceAccount: custom-scheduler
      containers:
        - name: custom-scheduler
          image: "<namespace>/<image name>:<tag>" (1)
          imagePullPolicy: Always

1	Specify the container image you created for the custom scheduler.

Deploying Pods using a Custom Scheduler

After your custom scheduler is deployed in your cluster, you can configure pods to use that scheduler instead of the default scheduler.

Create or edit a pod configuration and specify the name of the scheduler with the schedulerName parameter. The name must be unique.
Sample pod specification with scheduler
```
apiVersion: v1
kind: Pod
metadata:
  name: custom-scheduler-example
  labels:
    name: custom-scheduler-example
spec:
  schedulerName: custom-scheduler (1)
  containers:
  - name: pod-with-second-annotation-container
    image: docker.io/ocpqe/hello-pod
```
1 The name of the scheduler to use. When no scheduler name is supplied, the pod is automatically scheduled using the default scheduler.

Run the following command to create the pod:

$ oc create -f <file-name>.yaml

For example:

$ oc create -f custom-scheduler-example.yaml

Run the following command to check that the pod was created:

$ oc get pod <file-name>

For example:

$ oc get pod custom-scheduler-example

NAME                       READY     STATUS    RESTARTS   AGE
custom-scheduler-example   1/1       Running   0          4m

Run the following command to check that the custom scheduler scheduled the pod:

$ oc describe pod <pod-name>

For example:

$ oc describe pod custom-scheduler-example

The name of the scheduler is listed, as shown in the following truncated output:

...

Events:
  FirstSeen  LastSeen  Count  From                SubObjectPath  Type       Reason Message
  ---------  --------  -----  ----                -------------  --------   ------ -------
  1m         1m        1      custom-scheduler    Normal         Scheduled  Successfully assigned custom-scheduler to <$node1>

...