Triggering your Tekton Pipelines on a Github Push Event

Tekton Triggers

In a previous blog post we’ve used Tekton Pipelines to set up a simple pipeline that runs tests, builds a docker image and pushes it to a registry.

In this blog post we’re going to take a look at Tekton Triggers and integrate it with GitHub. We’re going to setup a GitHub webhook that will automatically run our pipeline when a PR is merged to the default branch (in our case it would be main branch).

Requirements

For this tutorial we need a Kubernetes cluster with an ingress-controller installed that can give us an external IP. In my demo I would be using the Openshift 4.8 cluster as it can give us an external IP with the help of Route.

We also need a GitHub repository where we can add the webhook.

Installation

Tekton Triggers requires Tekton Pipelines to be installed. We also need to install the core interceptors (GitHub, GitLab, BitBucket, and CEL) manifests as we’ll use them later on.

By default all resources will be installed in the tekton-pipelines namespace.

To install Tekton Pipelines refer to the installation guide. Once the Pipelines are installed successfully we can now move ahead to install the Triggers. To install Triggers on kubernetes you can run the following command

kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/previous/v0.15.1/release.yaml
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/previous/v0.15.1/interceptors.yaml

If you are using an OpenShift cluster then you need to run the following commands first before applying the above manifests

oc adm policy add-scc-to-user anyuid -z tekton-triggers-controller
oc adm policy add-scc-to-user anyuid -z tekton-triggers-core-interceptors
oc adm policy add-scc-to-user anyuid -z tekton-triggers-webhook

In my tutorial I am installing Pipelines and Triggers using upstream release but if you are an OpenShift user then you can install Tekton Pipelines and Triggers with the help of OpenShift Pipelines operator which is available in OperatorHub in order to reduce all the above steps. To install you can follow the installation guide.

Moving forward…

In this blog post I am going to trigger an API whenever there is a change in particular file and for that I need two tasks. First Task will check whether the file was present in the commit or not and second task which is going to make a CURL request to the API. The two tasks can be found below:

check-file-changed.yaml

The above task takes two parameters first is the commits array which comes from the json when payload of push event emitted by Github and second one is the file which we want to check whether changed or not. Now let’s apply it

$ kubectl create ns triggers-demo
$ kubectl -n triggers-demo apply --filename https://gist.githubusercontent.com/vinamra28/ff7852770a9b453135e534bde5ad818c/raw/c94956b53a48b9095af6ce4dcf3045be226ce847/check-file-changed.yaml

PS: please ignore the python script as I am bad at it 😅

The second Task can be installed by running the following command

$ tkn hub install task curl -n triggers-demo

We can now go ahead and create the Pipeline which will be triggered on an github push event.

triggers-pipeline.yaml

$ kubectl -n triggers-demo apply --filename https://gist.githubusercontent.com/vinamra28/46b18aac9dcf66559d0f30005534ab6a/raw/ab17b3423570647dfcbe2fb106e974bd7408db2f/triggers-pipeline.yaml

Now it’s time to create the resources related to Triggers

Creating Resources for Tekton Triggers

For our project we need to create the following resources:

• EventListener: A Kubernetes Service that listens for incoming HTTP requests and executes a Trigger.
• Trigger: Decides what to do with the received event. Sets a TriggerBinding, TriggerTemplate and Interceptor to run.
• TriggerBinding: Specifies the data to be extracted from the request and saved as parameters. This data will be passed to the TriggerTemplate.
• TriggerTemplate: A template of a resource (TaskRun/PipelineRun) to be created when an event is received.
• Interceptor: Processes an event to do custom validation or filtering.

Before we start with applying the Triggers manifests let’s first configure the RBAC. You can run the below to configure the RBAC:

$ kubectl -n triggers-demo --filename https://gist.githubusercontent.com/vinamra28/a9074c08f8b05a7fd657b682fdcb0e0d/raw/aba3ec62664b8511d1ab16c5310ddc494debe26c/triggers-demo-rbac.yaml

Note: Your RBAC configuration might vary depending on the permissions you are having.

We can now go ahead and start creating Trigger resources.

Triggers Flow

EventListener

An EventListener processes an incoming request and executes a Trigger.

Our EventListenerlooks like this:

apiVersion: triggers.tekton.dev/v1beta1
kind: EventListener
metadata:
  name: trigger-demo-el
spec:
  serviceAccountName: tekton-ci-listener
  triggers:
    - triggerRef: github-listener
  resources:
    kubernetesResource:
      serviceType: NodePort

After receiving the incoming request it will execute the github-listener Trigger.

Trigger

A Trigger decides what to do with the received event. Our Trigger looks like this:

apiVersion: triggers.tekton.dev/v1beta1
kind: Trigger
metadata:
  name: github-listener
spec:
  interceptors:
    - ref:
        name: "github"
      params:
        - name: "secretRef"
          value:
            secretName: ci-webhook
            secretKey: secret
        - name: "eventTypes"
          value: ["push"]
    - ref:
        name: "cel"
      params:
        - name: "filter"
          value: body.ref == 'refs/heads/trigger-config'
  bindings:
    - ref: github-binding
  template:
    ref: trigger-demo-template

Our github-listener trigger will then make use of Interceptor’s. An Interceptor let’s us validate or modify incoming requests before they trigger a PipelineRun.

GitHub Interceptor

The first interceptor we’re running is called github. It’s part of the core interceptors that we installed above. It makes sure that the request:

• has a valid format for GitHub webhooks
• matches a pre-defined secret (that we have set while setting the RBAC above)
• matches the push event type

The github interceptor requires a secret token. This token is set when creating the webhook in GitHub and will be validated by the github interceptor when the request arrives. Below is how the secret looks:

apiVersion: v1
kind: Secret
metadata:
  name: ci-webhook
type: Opaque
stringData:
  secret: "1234567"

CEL Interceptor

The second interceptor we’re using is called cel and is also included in the core interceptor manifests that we installed above. Interceptors are executed in the order they’re specified. The cel interceptor will run after the github interceptor.

It let’s us specify a CEL filter expression that will be applied to requests.

We’ll apply this filter expression because GitHub push events are sent for every action performed on any of the branch present in the configured github repository.

For this tutorial we only need to know when a PR was merged in trigger-config branch or a commit was pushed into that branch. If you want to read more about CEL expressions then you can refer the doc.

TriggerBinding

After the Trigger is done validating and modifying the incoming request, we need to extract values from it and bind them to variables that we can later use in our Pipeline. This is what a TriggerBinding is used for. Our TriggerBinding looks like this:

apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerBinding
metadata:
  name: github-binding
spec:
  params:
    - name: gitRepository
      value: $(body.repository.html_url)
    - name: commits
      value: $(body.commits)

We’re only interested in the following two fields out of which only one is necessary to perform the check and one is just to keep as annotation:

• gitRepository: The html URL of the repo which can be stored as annotation so that we can know from which repo the event happened.
• commits: the commits which were the part of push event.

The parameters are then passed to a TriggerTemplate.

TriggerTemplate

A TriggerTemplate is responsible for dynamically generating a resource. In our case it’s a PipelineRun.

The TriggerTemplate receives the two variables from the previously created TriggerBinding and makes them available under spec.resourceTemplates.

apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerTemplate
metadata:
  name: trigger-demo-template
  annotations:
    triggers.tekton.dev/old-escape-quotes: "true"
spec:
  params:
    - name: commits
      description: The commits which got pushed
    - name: gitRepository
      description: The git repository that hosts context
  resourcetemplates:
    - apiVersion: tekton.dev/v1beta1
      kind: PipelineRun
      metadata:
        generateName: config-refresh-run-
        annotations:
          tekton.dev/gitURL: "$(tt.params.gitRepository)"
      spec:
        pipelineRef:
          name: config-refresh-pipeline
        params:
          - name: commits
            value: "$(tt.params.commits)"
          - name: filename
            value: "config.yaml"
          - name: urlToCurl
            value: https://api.hub.tekton.dev/categories
          - name: options
            value:
              ["-i", "-k", "-X", "GET"]

Note that to access variables from a triggertemplate inside a resourcetemplate they need to be prefixed with $tt.

Ingress

For GitHub to be able to send a request to our event listener we need to expose it by creating an Ingress resource and pointing it to our event listener service:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
    - http:
        paths:
          - path: /hooks
            pathType: Exact
            backend:
              service:
                name: el-trigger-demo-el
                port:
                  number: 8080

An EventListener will create a service with the el- prefix followed by the name of the event-listener. Our event-listener is named trigger-demo-el, so the EventListener Service is named el-trigger-demo-el. EventListener services will always use port 8080.

Make sure to note the external IP address of your ingress. In this example it’s 123.123.1.1:

$ kubectl get ingress
NAME               CLASS    HOSTS   ADDRESS        PORTS   AGE
ingress-resource   <none>   *       123.123.1.1    80      26d

Route

If you are using OpenShift then you can create a route by exposing the service by running the command oc expose svc/el-trigger-demo-el . You can then get the route by running the command oc get route .

We can now send GitHub webhook requests to http://123.123.1.1/hooks or the route obtained from Openshift and automatically trigger a PipelineRun.

Adding the webhook to Github

In your GitHub repo go to Settings -> Webhooks and click Add Webhook. The fields we need to set are:

• Payload URL: Your external IP Address from the Ingress with /hooks path or the OpenShift route.
• Content type: application/json
• Secret: 1234567 which should match the secret value created above.

Under events select Just the push event.

After saving the webhook, GitHub will send a ping event. It will be filtered out by our Interceptor which only allows push events, but we can check the EventListener Pod logs to verify it by running

$ kubectl get pods | grep el-trigger-demo-el
el-trigger-demo-el-1df7f66d57-tqe6k     1/1     Running   0          13s
$ kubectl logs el-trigger-demo-el-1df7f66d57-tqe6k
...
{
  "level": "info",
  "ts": "2021-08-29T08:48:59.461Z",
  "logger": "eventlistener",
  "caller": "sink/sink.go:240",
  "msg": "interceptor stopped trigger processing: rpc error: code = FailedPrecondition desc = event type ping is not allowed",
  "knative.dev/controller": "eventlistener",
  "/triggers-eventid": "f43a11d8-a462-47c9-a2cd-f98de681da40",
  "/trigger": "github-listener"
}

Creating a branch and testing the Trigger

Let’s test that everything works by creating a branch named trigger-config, either manually or using the GitHub CLI:

git switch -c trigger-config
git commit --allow-empty -m "trigger webhook"
git push origin trigger-config

And checking for a PipelineRun to get created:

$ kubectl get pr
NAME                           SUCCEEDED   REASON      STARTTIME   COMPLETIONTIME
config-refresh-run-qfvsx       True        Succeeded   73s         44s

We can see that everything worked. The pipeline was triggered and the run succeeded.

Questions?

Follow and hit me up on Twitter @jvinamra776 if you have any questions or comments. If I’ve made a mistake or if there is something that I have missed out in this article be sure to let me know so that I can get it corrected.

Do follow me up on medium for more Tekton related stuffs 😉.