May 08, 2021 - 5 min read
The full code described in this post can be found on Github.
As a learning exercise, I wanted to see how I could put together a pipeline for a mono-repo with Tekton. This post outlines a setup that supports the execution of multiple pipelines from a single Github push event, learning a little about Tekton interceptors along the way.
Tekton is an open-source framework for creating CI/CD systems. Tekton uses Kubernetes Custom Resource Definitions to define pipelines, and orchestrates the pipeline stages as Kubernetes pods.
Typically, a CD pipeline would be initiated by a Git webhook (i.e a Github push event), triggering a pipeline to run for the associated Git repository. These
pipelines are usually defined in YAML (e.g Gitlab’s
.gitlab-ci.yaml, or Travis’
Most CI/CD tools have the following features in common:
Modelling a mono-repository with the above features has a few issues:
Using Tekton’s building blocks, I wanted to see if I could trigger a different pipeline depending on the sub-directories that were modified for a given Git event.
Triggers is the sub-project of Tekton that is responsible for triggering pipelines in response to external events via webhooks. Tekton’s own documentation gives a good overview.
EventListeners expose an addressable “Sink” to which incoming events are directed. Users can declare TriggerBindings to extract fields from events, and apply them to TriggerTemplates in order to create Tekton resources. In addition, EventListeners allow lightweight event processing using Event Interceptors.
At a high level, an
EventListener defines the endpoint for incoming events, passes the events off to a list of
Trigger resources. These
Trigger resources define how to map the event to a given pipeline’s input parameters.
Trigger can define interceptors that allow you to process webhook payloads before any pipeline gets triggered, potentially deciding not to trigger any pipeline at all.
Tekton comes bundled with some out of the box interceptors:
In the pipeline setup described by this post, there’s a single
EventListener that references a unique
Trigger for each sub-project within the repo. Each
Trigger references a
TriggerTemplate configured to create
PipelineRun for each sub-project’s
The mono-repo magic is handled by the interceptors on each
Trigger. These interceptors pre-process the incoming Github events, conditionally triggering each project’s pipeline only if that project’s directory is modified. If one project directory is modified then only that project’s pipeline will run. If both are modified then both project pipelines will run.
Below is Project A’s
apiVersion: triggers.tekton.dev/v1alpha1 kind: Trigger metadata: name: service-a-trigger spec: interceptors: - webhook: objectRef: kind: Service name: interceptor apiVersion: v1 namespace: default - cel: filter: body.extensions.filesChanged.exists(i, i.startsWith('service-a/')) bindings: - name: service value: service-a - name: pipeline-name value: service-a-pipeline template: ref: mono-trigger-template
As can be seen above, the
Trigger defines two interceptors:
interceptorrunning within the cluster. This service uses Github’s Compare API to fetch the list of modified files between two commits, adding them to into the JSON payload at
filterpredicate that will conditionally trigger the pipeline. This interceptor defines a predicate that only passes if the list of modified paths contains a file related to its associated sub-project,
service-a/in the case of the above
Multiple related projects can be co-located in the same repository without having to share a single definition. This keeps the pipeline definitions for each project simple, specific and with a single responsibility. As a result, it’s easy to find the pipeline execution for a given sub-project using either the Tekton Dashboard or
kubectl as they are unique pipelines.
Additionally, adding a
labels fields to the
metadata field of a
Pipeline resource means that pipelines for a given service or sub-project can be filtered for easily.
Tekton provides more flexibility compared to other pipeline tools at the cost of some additional complexity. Tekton provides the building blocks for building a CI/CD system - you need to stitch those blocks together to suite your needs.
Whilst I wouldn’t say the setup is very complex, without some familiarity of Tekton the process of adding a new sub-project isn’t super simple. As Tekton pipelines are defined as plain old Kubernetes CRDs they can be distributed the same way as an application might be, Helm.
Below is a the
values.yaml of a Helm chart written to wrap the trigger setup described in this post.
# The name of the repository this trigger is created for. Used to name resources. repositoryName: tekton-mono-repo-demo # An Ingress is created for this endpoint. e.g /my-repo-trigger webhookEndpoint: /tekton-mono-repo-demo # The name of the service account that the event listener runs under. serviceAccount: ... # A list of the projects within the repository mapped to the pipeline they should trigger. projects: - name: project-a path: project-a/ pipeline: project-a-pipeline - name: project-b path: project-b/ pipeline: project-b-pipeline
I think this demonstrates that Helm provides a great way to reuse and distribute Tekton pipeline best practices. Deploying a Helm chart or modifying its values is objectively more simple than needing to understand the building blocks of Tekton.