Build a Kubernetes Operator using Operator SDK
This post will guide you to create a Kubernetes operator using Operator SDK.
Why Kubernetes Operator?
Let’s say you have an application with multiple services when you want to deploy a new instance you have to manually deploy all services or you can use a script. But if you use an operator then this will do your job to deploy all services and also if a new version is released it can update the application to the newer version.
In short, you can embed all the human logic into the operator and the operator will manage your whole application lifecycle.
You can share the operator with everyone and it would be quite easy to get an instance of the application running.
If you embed the logic into the operator then it can do your job to manage your application i.e. creating, updating, etc.
Why Operator SDK?
The Operator SDK makes easy building Kubernetes applications. It provides high-level APIs, useful abstractions, and project scaffolding. It will create a boilerplate code and then we can embed our logic into it.
You can read more about it here.
We will build an operator that will deploy a Hello world Application, and expose it to access it from outside cluster.
You can find the source code for the operator here.
Prerequisites:
- Golang: we will use Golang to build the operator. You can install it from here. Also, configure the GOPATH
$GOPATH=/your/preferred/path/
$GO111MODULE=on
- Operator SDK: Install the SDK. you can find the installation steps here. I will be using SDK v0.1 which is recently released.
- Access to Kubernetes cluster
Let’s get started.
Create a new project
$ mkdir -p hello-operator
$ cd hello-operator# we'll use a domain of hello.com
# so all API groups will be <group>.hello.com$ operator-sdk init --domain=hello.com --repo=github.com/sm43/hello-operator
The path to my project is /go/src/github.com/sm43/hello-operator
This command generates the boilerplate code for our operator. Now, the directory structure would look as below.
.
├── bin
├── config
│ ├── certmanager
│ ├── default
│ ├── manager
│ ├── prometheus
│ ├── rbac
│ ├── scorecard
│ │ ├── bases
│ │ └── patches
│ └── webhook
└── hack
Create a new API and Controller
$ operator-sdk create api --group=init --version=v1alpha1 --kind=Hello --controller --resource
This command will create a new Custom Resource Definition(CRD) API with a group init
version v1alpha1
and Kind Hello
.
Define the API
Now, we will define the Hello Custom Resource (CR) what spec it will have to create the resource. we need to edit api/v1alpha1/hello_types.go
to define the Hello CR.
// HelloSpec defines the desired state of Hello
type HelloSpec struct {
Image string `json:"image"`
Replica int32 `json:"replica"`
}// HelloStatus defines the observed state of Hello
type HelloStatus struct {
Status string `json:"status"`
}
Here, HelloSpec
will have 2 fieldsimage
where we will pass the image name and replica
where we can say the number replicas of application we require.
HelloStatus
will have the status of the resource once it is created. The operator will update this field according to the status of the resource so that the user will get to know.
After modifying the hello_types.go
file always run the following command to update the generated code for that resource type:
$ make generate
Generate CRD manifests
Once you are done defining the CR run the below command.
$ make manifests
After running the command, you can check a new file will be created config/crd/bases/init.hello.com_hellos.yaml
. This is the Custom Resource Definition manifest.
Always run the commands after updating the CR in hello_types.go
Implement the Controller
You can find the whole code for the controller here.
The controller executes the logic as below:
Let’s say there is an instance of Hello
CR created by name hey
.
- The controller will get a request for
hey
- It will check if a deployment is created with the name
hey
, if not then it will create a deployment with the image and number of replicas defined in YAML used for creatinghey
. - It will also create
Service
andRoute
with the namehey
so that the application can be exposed outside the cluster.
NOTE: Route
is a concept in OpenShift, where you can easily create an URL for your application. If you don’t have an OpenShift cluster don’t worry go on, we will figure out a way in the next part to use this Operator in any Kubernetes cluster.
- If the deployment is already created then it checks if the number of replicas requested is different than the current one. If they are different then it will update the replicas to the desired state.
- And then It will update the status of
hey
resource saying tohey Hello CR running with <no.of desired replicas> replicas
.
Build and run the Operator
The controller needs certain RBAC permissions to interact with the resources it manages. It is defined in the controller as
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete
The ClusterRole
manifest at config/rbac/role.yaml
is generated from the above markers via controller-gen with the following command:
$ make manifests
Now is the time to install CR and run the Operator. If you don’t have an OpenShift cluster you can get one from https://www.openshift.com/try. You can also run an OpenShift cluster on your local machine, follow the steps mentioned here.
If you want to try this on a Kubernetes cluster then comment on the code where it creates route, as Kubernetes doesn’t have any idea what route is, it will throw an error. Go in controllers/hello_controller.go
and comment L92 to L99. This is where it creates a route on the cluster.
So, the service we are creating is a NodePort
so to access this outside your Kubernetes cluster create a firewall rule that allows TCP traffic on your node port once created.
Make sure you are logged into your cluster.
Before running the operator, the CRD must be registered with the Kubernetes API server:
$ make install
You can see the installed CRD using the below command
kubectl get crd | grep hello
There are 2 ways to run the Operator
- Run locally outside the cluster
- Run as a Deployment inside the cluster
While developing the Operator running locally is a good option as you can test the code as you are developing.
To create an image and run as deployment you can follow the steps mentioned here.
To run the operator locally execute the following command:
$ make run ENABLE_WEBHOOKS=false
Now, we can create a Hello CR. There is a sample CR in config/samples
. You can edit init_v1alpha1_hello.yaml
and apply the file using the below command
$ kubectl apply -f config/samples/init_v1alpha1_hello.yaml
You can now check the deployment, pods, service, and route created by it.
Access the route using the below command
curl $(kubectl get routes hey --template='http://{{ .spec.host }}')
and you would be able to see
Hello Kubernetes!
So, now you have an idea of how we automated the process of creating multiple resources using an operator, just by applying the CR sample you can create all the required resources for the app.
You can find the source code for the operator here.
Reach out to me on GitHub(SM43)/LinkedIn, we will build stuff together ;)
References: