Build a Kubernetes Operator using Operator SDK

This post will guide you to create a Kubernetes operator using Operator SDK.

Why Kubernetes Operator?

Let’s say you have an application with multiple services when you want to deploy a new instance you have to manually deploy all services or you can use a script. But if you use an operator then this will do your job to deploy all services and also if a new version is released it can update the application to the newer version.
In short, you can embed all the human logic into the operator and the operator will manage your whole application lifecycle.

You can share the operator with everyone and it would be quite easy to get an instance of the application running.

If you embed the logic into the operator then it can do your job to manage your application i.e. creating, updating, etc.

Why Operator SDK?

The Operator SDK makes easy building Kubernetes applications. It provides high-level APIs, useful abstractions, and project scaffolding. It will create a boilerplate code and then we can embed our logic into it.
You can read more about it here.

We will build an operator that will deploy a Hello world Application, and expose it to access it from outside cluster.

You can find the source code for the operator here.

Prerequisites:

• Golang: we will use Golang to build the operator. You can install it from here. Also, configure the GOPATH

$GOPATH=/your/preferred/path/
$GO111MODULE=on

• Operator SDK: Install the SDK. you can find the installation steps here. I will be using SDK v0.1 which is recently released.
• Access to Kubernetes cluster

Let’s get started.

Create a new project

$ mkdir -p hello-operator
$ cd hello-operator
# we'll use a domain of hello.com
# so all API groups will be <group>.hello.com
$ operator-sdk init --domain=hello.com --repo=github.com/sm43/hello-operator

The path to my project is /go/src/github.com/sm43/hello-operator

This command generates the boilerplate code for our operator. Now, the directory structure would look as below.

.
├── bin
├── config
│   ├── certmanager
│   ├── default
│   ├── manager
│   ├── prometheus
│   ├── rbac
│   ├── scorecard
│   │   ├── bases
│   │   └── patches
│   └── webhook
└── hack

Create a new API and Controller

$ operator-sdk create api --group=init --version=v1alpha1 --kind=Hello --controller --resource

This command will create a new Custom Resource Definition(CRD) API with a group init version v1alpha1 and Kind Hello.

Define the API

Now, we will define the Hello Custom Resource (CR) what spec it will have to create the resource. we need to edit api/v1alpha1/hello_types.go to define the Hello CR.

// HelloSpec defines the desired state of Hello
type HelloSpec struct {
    Image   string `json:"image"`
    Replica int32  `json:"replica"`
}
// HelloStatus defines the observed state of Hello
type HelloStatus struct {
    Status string `json:"status"`
}

Here, HelloSpec will have 2 fieldsimage where we will pass the image name and replica where we can say the number replicas of application we require.

HelloStatus will have the status of the resource once it is created. The operator will update this field according to the status of the resource so that the user will get to know.

After modifying the hello_types.go file always run the following command to update the generated code for that resource type:

$ make generate

Generate CRD manifests

Once you are done defining the CR run the below command.

$ make manifests

After running the command, you can check a new file will be created config/crd/bases/init.hello.com_hellos.yaml . This is the Custom Resource Definition manifest.

Always run the commands after updating the CR in hello_types.go

Implement the Controller

You can find the whole code for the controller here.

The controller executes the logic as below:

Let’s say there is an instance of Hello CR created by name hey.

• The controller will get a request for hey
• It will check if a deployment is created with the name hey , if not then it will create a deployment with the image and number of replicas defined in YAML used for creating hey.
• It will also create Service and Route with the name heyso that the application can be exposed outside the cluster.

NOTE: Route is a concept in OpenShift, where you can easily create an URL for your application. If you don’t have an OpenShift cluster don’t worry go on, we will figure out a way in the next part to use this Operator in any Kubernetes cluster.

• If the deployment is already created then it checks if the number of replicas requested is different than the current one. If they are different then it will update the replicas to the desired state.
• And then It will update the status of hey resource saying to hey Hello CR running with <no.of desired replicas> replicas.

Build and run the Operator

The controller needs certain RBAC permissions to interact with the resources it manages. It is defined in the controller as

//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete

The ClusterRole manifest at config/rbac/role.yaml is generated from the above markers via controller-gen with the following command:

$ make manifests

Now is the time to install CR and run the Operator. If you don’t have an OpenShift cluster you can get one from https://www.openshift.com/try. You can also run an OpenShift cluster on your local machine, follow the steps mentioned here.

If you want to try this on a Kubernetes cluster then comment on the code where it creates route, as Kubernetes doesn’t have any idea what route is, it will throw an error. Go in controllers/hello_controller.go and comment L92 to L99. This is where it creates a route on the cluster.

So, the service we are creating is a NodePort so to access this outside your Kubernetes cluster create a firewall rule that allows TCP traffic on your node port once created.

Make sure you are logged into your cluster.

Before running the operator, the CRD must be registered with the Kubernetes API server:

$ make install

You can see the installed CRD using the below command

kubectl get crd | grep hello

There are 2 ways to run the Operator

1. Run locally outside the cluster
2. Run as a Deployment inside the cluster

While developing the Operator running locally is a good option as you can test the code as you are developing.

To create an image and run as deployment you can follow the steps mentioned here.

To run the operator locally execute the following command:

$ make run ENABLE_WEBHOOKS=false

Now, we can create a Hello CR. There is a sample CR in config/samples. You can edit init_v1alpha1_hello.yaml and apply the file using the below command

$ kubectl apply -f config/samples/init_v1alpha1_hello.yaml

You can now check the deployment, pods, service, and route created by it.

Access the route using the below command

curl $(kubectl get routes hey --template='http://{{ .spec.host }}')

and you would be able to see

Hello Kubernetes!

So, now you have an idea of how we automated the process of creating multiple resources using an operator, just by applying the CR sample you can create all the required resources for the app.

You can find the source code for the operator here.

Reach out to me on GitHub(SM43)/LinkedIn, we will build stuff together ;)

References:

• Operator SDK Docs