Extending the Kubernetes API with custom resources
In the Kubernetes API a resource is an endpoint that stores a collection of API objects of a certain kind. For example, the built-in pods resource contains a collection of Pod objects.
A custom resource is an object that extends the Kubernetes API or allows you to introduce your own API into a project or a cluster.
A custom resource definition (CRD) file defines your own object kinds and lets the API Server handle the entire lifecycle. Deploying a CRD into the cluster causes the Kubernetes API server to begin serving the specified custom resource.
When you create a new custom resource definition (CRD), the Kubernetes API Server reacts by creating a new RESTful resource path, that can be accessed by an entire cluster or a single project (namespace). As with existing built-in objects, deleting a project deletes all custom objects in that project.
If you want to grant access to the CRD to users, use cluster role aggregation to grant access to users with the admin, edit, or view default cluster roles. Cluster role aggregation allows the insertion of custom policy rules into these cluster roles. This behavior integrates the new resource into the cluster’s RBAC policy as if it was a built-in resource.
While only cluster admins can create CRDs, you can create an object from a CRD if you have read and write permission to it.
To create custom objects, you must first create a custom resource definition (CRD).
Only cluster admins can create CRDs.
To create a CRD:
Create a YAML file that contains the fields in the following example:Example YAML file for a Custom Resource Definition
apiVersion: apiextensions.k8s.io/v1beta1 (1) kind: CustomResourceDefinition metadata: name: crontabs.stable.example.com (2) spec: group: stable.example.com (3) version: v1 (4) scope: Namespaced (5) names: plural: crontabs (6) singular: crontab (7) kind: CronTab (8) shortNames: - ct (9)
1 Use the
2 Specify a name for the definition. This must be in the <plural-name><group> format using the values from the
3 Specify a group name for the API. An API group is a collection of objects that are logically related. For example, all batch objects like
ScheduledJobcould be in the batch API Group (such as batch.api.example.com). A good practice is to use a fully-qualified-domain name of your organization.
4 Specify a version name to be used in the URL. Each API Group can exist in multiple versions. For example:
5 Specify whether the custom objects are available to a project (
Namespaced) or all projects in the cluster (
6 Specify the plural name to use in the URL. The
pluralfield is the same as a resource in an API URL.
7 Specify a singular name to use as an alias on the CLI and for display. 8 Specify the kind of objects that can be created. The type can be in CamelCase. 9 Specify a shorter string to match your resource on the CLI.
By default, a custom resource definition is cluster-scoped and available to all projects.
Create the object:
oc create -f <file-name>.yaml
A new RESTful API endpoint is created at:
For example, using the example file, the following endpoint is created:
You can use this endpoint URL to create and manage custom objects. The kind of object is based on the
spec.kindfield of the Custom Resource Definition object you created.
After you create a cluster-scoped custom resource definition (CRD), you can grant permissions to it. If you use the admin, edit, and view default cluster roles, take advantage of cluster role aggregation for their rules.
You must explicitly assign permissions to each of these roles. The roles with more permissions do not inherit rules from roles with fewer permissions. If you assign a rule to a role, you must also assign that verb to roles that have more permissions. For example, if you grant the "get crontabs" permission to the view role, you must also grant it to the edit and admin roles. The admin or edit role is usually assigned to the user that created a project through the project template.
Create a CRD.
Create a cluster role definition file for the CRD. The cluster role definition is a YAML file that contains the rules that apply to each cluster role. The OKD controller adds the rules that you specify to the default cluster roles.Example YAML file for a cluster role definition
kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 (1) metadata: name: aggregate-cron-tabs-admin-edit (2) labels: rbac.authorization.k8s.io/aggregate-to-admin: "true" (3) rbac.authorization.k8s.io/aggregate-to-edit: "true" (4) rules: - apiGroups: ["stable.example.com"] (5) resources: ["crontabs"] (6) verbs: ["get", "list", "watch", "create", "update", "patch", "delete", "deletecollection"] (7) --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: aggregate-cron-tabs-view (2) labels: # Add these permissions to the "view" default role. rbac.authorization.k8s.io/aggregate-to-view: "true" (8) rbac.authorization.k8s.io/aggregate-to-cluster-reader: "true" (9) rules: - apiGroups: ["stable.example.com"] (5) resources: ["crontabs"] (6) verbs: ["get", "list", "watch"] (7)
1 Use the
2 Specify a name for the definition. 3 Specify this label to grant permissions to the admin default role. 4 Specify this label to grant permissions to the edit default role. 5 Specify the group name of the CRD. 6 Specify the plural name of the CRD that these rules apply to. 7 Specify the verbs that represent the permissions that are granted to the role. For example, apply read and write permissions to the admin and edit roles and only read permission to the view role. 8 Specify this label to grant permissions to the view default role. 9 Specify this label to grant permissions to the cluster-reader default role.
Create the cluster role:
oc create -f <file-name>.yaml
After you create the custom resource definition (CRD) object, you can create custom objects that use its specification.
Custom objects can contain custom fields that contain arbitrary JSON code.
Create a CRD.
Create a YAML definition for the custom object. In the following example definition, the
imagecustom fields are set in a custom object of kind
CronTab. The kind comes from the
spec.kindfield of the custom resource definition object.Example YAML file for a custom object
apiVersion: "stable.example.com/v1" (1) kind: CronTab (2) metadata: name: my-new-cron-object (3) finalizers: (4) - finalizer.stable.example.com spec: (5) cronSpec: "* * * * /5" image: my-awesome-cron-image
1 Specify the group name and API version (name/version) from the custom resource definition. 2 Specify the type in the custom resource definition. 3 Specify a name for the object. 4 Specify the finalizers for the object, if any. Finalizers allow controllers to implement conditions that must be completed before the object can be deleted. 5 Specify conditions specific to the type of object.
After you create the object file, create the object:
oc create -f <file-name>.yaml
After you create objects, you can manage your custom resources.
Create a custom resource definition (CRD).
Create an object from a CRD.
To get information on a specific kind of custom resource, enter:
oc get <kind>
oc get crontab NAME KIND my-new-cron-object CronTab.v1.stable.example.com
Note that resource names are not case-sensitive, and you can use either the singular or plural forms defined in the CRD, as well as any short name. For example:
oc get crontabs oc get crontab oc get ct
You can also view the raw YAML data for a custom resource:
oc get <kind> -o yaml
oc get ct -o yaml apiVersion: v1 items: - apiVersion: stable.example.com/v1 kind: CronTab metadata: clusterName: "" creationTimestamp: 2017-05-31T12:56:35Z deletionGracePeriodSeconds: null deletionTimestamp: null name: my-new-cron-object namespace: default resourceVersion: "285" selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object uid: 9423255b-4600-11e7-af6a-28d2447dc82b spec: cronSpec: '* * * * /5' (1) image: my-awesome-cron-image (1)
1 Custom data from the YAML that you used to create the object displays.