Tekton: Building a Pipeline (Part 4)

This article is part of Series World of Tekton and illustrate how to build and run a Pipeline.

This is Part 4 of the series, you can find the introduction here :)

World of Tekton 😺 (Part 1)

In this article, we will build and run a Tekton Pipeline. We will using a news-demo app written in go.
You can find it in the repository https://github.com/sm43/news-demo. It is forked from Freshman-tech/news-demo, and modified a bit to deploy on Kubernetes.

Prerequisite:

• Kubernetes Cluster
  I will be using an OpenShift cluster but any you can follow with any K8s cluster.
• kubectl CLI
• tkn CLI (Optional)

# Install Tekton Pipeline

If you are using OpenShift Cluster then you can go to Operator Hub and install Red Hat OpenShift Pipeline Operator.

Installation

If you are using any other Kubernetes cluster, then you can install using Tekton Operator.

You can find the installation steps here.

# Deploy News Demo Application

Before looking into the Pipeline, let’s first deploy the application on the cluster.

You can clone the repository using

git clone https://github.com/sm43/news-demo.git

The application requires a News API account and its key to work. Sign up for a News API account and get your free API key.

Once you have the key

cd news-demo

Edit the configMap in k8s directory and add your API key for the variable NEWS_API_KEY.

After updating the configMap, you can apply the manifest on the cluster using kubectl.

kubectl apply -f k8s/

This will deploy the application in news-demo namespace and create a service for the deployment.

To access the application outside cluster, Create a Route if you are OpenShift cluster.

kubectl apply -f k8s/openshift/

If you are using kubenetes cluster then you need to create an Ingress to access the application outside the cluster.

You can get the route and access it on a browser.

echo "https://$(kubectl get routes news-demo -n news-demo -ojsonpath='{.spec.host}')"

# Setting up a Pipeline

For the news demo app, we will setting up a Pipeline which would be doing following tasks:

We will be reusing tasks from Tekton Catalog, you can browse them on hub.tekton.dev.

• fetch code
  for fetching the code, we have git-clone Task. This task takes repository url, branch name as input and clones the code in a workspace which we would be passing to it.
• build test
  for building the code, we have goland-buildTask. This would build the code and Task would fail if the build fails.
• unit test
  to run the unit test, we have golang-test Task.
• build and push image
  we have dockerfile in the repo to build an image for the application. we have buildah Task which builds the image and push to an image registry
• checking/creating/updating deployment
  once we have a new image for the application, we check if the deployment exist, if its doesn’t then we create a new deployment or we patch the existing deployment with the new image. For this we have kubernetes-action Task.

If we bring all the Tasks together then the pipeline would look like …
https://github.com/sm43/news-demo/blob/main/pipeline/01-pipeline.yaml

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: news-demo-deploy
spec:
  # params to be passed from pipelineRun 
  params:
    - name: REPO
    - name: REVISION
    - name: IMAGE
    - name: TAG
    - name: NAMESPACE
  
  # workspace to be passed from pipelineRun in which repo will
  # be cloned and shared among all Tasks
  workspaces:
    - name: shared-workspace
  
  tasks:    
    - name: fetch-repository
      taskRef:
        name: git-clone
      workspaces:
        - name: output
          workspace: shared-workspace
      params:
        - name: url
          value: $(params.REPO)
        - name: subdirectory
          value: ""
        - name: deleteExisting
          value: "true"
        - name: revision
          value: $(params.REVISION)
    - name: build-test
      runAfter:
        - fetch-repository
      taskRef:
        name: golang-build
      params:
        - name: packages
          value: ./...
        - name: package
          value: github.com/sm43/news-demo
      workspaces:
        - name: source
          workspace: shared-workspace
    - name: unit-tests
      runAfter:
        - fetch-repository
      taskRef:
        name: golang-test
      params:
        - name: package
          value: github.com/sm43/news-demo
        - name: flags
          value: -v -mod=vendor
      workspaces:
        - name: source
          workspace: shared-workspace
    - name: build-push-image
      taskRef:
        name: buildah
      workspaces:
        - name: source
          workspace: shared-workspace
      params:
        - name: IMAGE
          value: $(params.IMAGE):$(params.TAG)
        - name: FORMAT
          value: "docker"
      runAfter:
        - build-test
        - unit-tests
    - name: check-deployment
      taskRef:
        name: kubernetes-actions
      params:
        - name: script
          value: |
            kubectl describe deployment news-demo -n "$(params.NAMESPACE)" >/dev/null 2>/dev/null
            if [[ $? -eq 0 ]]; then
              printf yes | tee /tekton/results/output-result
            else
              printf no | tee /tekton/results/output-result
            fi
      runAfter:
        - build-push-image
    - name: patch-image
      taskRef:
        name: kubernetes-actions
      params:
        - name: script
          value: |
            kubectl patch deployment news-demo --patch='{"spec":{"template":{"spec":{
              "containers":[{
                "name": "news-demo",
                "image": "$(params.IMAGE):$(params.TAG)"
              }]
            }}}}' -n $(params.NAMESPACE)
      when:
        - input: "$(tasks.check-deployment.results.output-result)"
          operator: in
          values: ["yes"]
      runAfter:
        - check-deployment
    - name: create-deployment
      taskRef:
        name: kubernetes-actions
      workspaces:
        - name: manifest-dir
          workspace: shared-workspace
      params:
        - name: script
          value: |
            kubectl -n $(params.NAMESPACE) apply -f <(sed "s@image:.*@image: $(params.IMAGE):$(params.TAG)@" k8s/02-deployment.yaml)
      when:
        - input: "$(tasks.check-deployment.results.output-result)"
          operator: in
          values: ["no"]

let’s understand the pipeline …

pipeline flow

• first task clone the repository inside a workspace, the workspace we would be using is a pvc which will passed to the pipeline from PipelineRun
• after completing the first task, build test and unit test start parallely.
• but the next task build-push-image will execute only if previous are successful
• if any of the build-test or unit-test fails, then the pipeline fails
• the order is specified by `runAfter` field descibed in each task in the above pipeline
• after pushing the image, we check if deployment for news-demo already exist on the cluster
• if it doesn’t exist, then we create a new deployment
• or we just patch the existing deployment with new image

# Execution

We have installed Tekton using the Tekton Operator on Kubernetes or OpenShift Pipelines Operator on OpenShift.

Before applying the pipeline, we will need to create some resources, run the bash script in pipeline directory.

Edit the script to add your image registry credentials so that pipeline can push image to your registry. and update your registry username in pipelineRun.

Execute the scipt

./pipeline/run.sh

This script install task from catalog, create service account which has access to your registry for pushing the image, rbac required for creating/updating deployment, pipeline and starts the pipeline by creating pipelineRun.

You can use tkn to access the resource

# List the pipelineRuns
tkn pipelinerun ls
# follow the logs of pipelineRun
tkn pipelinerun logs -f

Wait for pipelineRun to be completed and then check the image in deployment..

Previously, when we deployed the application the image was `quay.io/sm43/news-demo:latest` and now the pipeline has updated the image to `quay.io/sm43/news-demo:v0.1`.

We have successfully ran the pipeline.

In next part, we will setup automated triggering of Pipeline using Tekton Triggers.

NEXT:

Tekton: Triggering the Pipeline (Part 5)

PREV:

Tekton: Concepts of Triggers (Part 3)