Gitlab CI/CD Serverless Monorepo

Is your repository full of disparate serverless functions which you want to integrate CI/CD into? Look no further, here’s my quick and easy guide on configuring a CI/CD pipeline for your functions repository.

Implementing a monorepo strategy can be benefitial if you have a large number of functions that do not belong to a single big feature, creating a new repo for each one creates a huge overhead and requires you to manage/duplicate credentials/permissions/user access/env variable (for the CICD) in multiple places.

So the easy way to do it:

---
image: amaysim/serverless:1.79.0
stages:
- deploy
variables:
# AWS region to deploy function
REGION: ap-southeast-1
.deploy-serverless-aws:
stage: deploy
script:
- cd services/$WORKDIR/
- serverless deploy --stage $ENV --region $REGION
deploy-app1:
extends:
- .deploy-serverless-aws
variables:
WORKDIR: app1
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
changes:
- services/app1/

The first job is a “template” job for a basic serverless deployment to AWS lambda. Then you can add a simple job definition for each function you wish to deploy which extends the original job.

You can create multiple templates (e.g. deploy GCP functions instead, or different build/test/deploy steps for a different programming language).

More examples:

---
image: amaysim/serverless:1.79.0
stages:
- deploy
variables:
# default unit test command
TEST: "npm run test"
# AWS region to deploy function
REGION: ap-southeast-1
.deploy-serverless-aws:
stage: deploy
before_script:
- |
cd services/$WORKDIR/
sls plugin install -n serverless-python-requirement
export ENV=$([[ $CI_COMMIT_BRANCH == "master" ]] && echo "prod" || echo "dev")
eval "$TEST"
script:
- serverless deploy --stage $ENV --region $REGION
deploy-app1:
extends:
- .deploy-serverless-aws
variables:
WORKDIR: app1
TEST: |
python3 -m pip install pytest && \
python3 -m pip install -r requirements.txt && \
python3 -m pytest -v --disable-warnings
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
changes:
- services/app1/
# deploy-another-function:
# extends: .deploy-serverless-aws
# variables:
# WORKDIR: another-function

This template is deploying node functions by default, which will invoke npm run test before deploying. But we can turn the test command into a variable and override from a specific function CI/CD job. Additionally each commit in a non master branch (e.g. feature branch or dev branch) will automatically deploy to a dev environment, commit on master will deploy to production.

Limitations

Due to limitations on Gitlab CI and how we can design the pipeline to work for a monorepo, we include the unit/integration testing as a pre-deployment step instead of a different CI job. And we duplicate the rules definition for each job as we cannot expand the $WORKDIR variable when evaluating changes.

Changes keyword does not seem to accept directory globbing, in which case, you must specify up to an arbitrary number of directory depthness you want to check for changes. My Gitlab issue: https://forum.gitlab.com/t/gitlab-ci-glob-rules-changed-files/44663 What you need to do instead:

only:
changes:
- "services/app/**"
- "services/app/**/*"
- "services/app/**/**/*"
- "services/app/**/**/**/*"

When working with feature branches, when merging master branch to your code, it will trigger a deployment for any changed function that was updated with code from master branch. That is, while your branch, only shows a diff for the function you working on, Gitlab CI will deploy all functions touched by merge master. See my filled issue https://gitlab.com/gitlab-org/gitlab/-/issues/295288

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store