7.9 C
London
Friday, March 29, 2024

Automated Versioning of Terraform Modules with GitHub Actions

Co-authored with the amazing Daniel Lamando

Using modules to keep your Terraform code DRY is very common, but figuring out the best way to release and distribute these modules is not always easy. We at Forto quickly faced this struggle, as more of the organization adopted Terraform to describe their infrastructure with code. In fact, keen-eyed readers of our previous Terraform blog post might have noticed a small easter egg:

Do not version your Terraform modules like that indeed!

This snippet represented how we referenced Terraform modules at the time, and it exposed some significant technical debt: referencing unversioned Terraform modules by-path. This practice made every change to any module a risky endeavor. In some cases, we created extra copies of our modules to manage the risk level (leading to @ v2 as seen in that screenshot).

When we realized the need to version our Terraform modules, I planned a hack day to find a solution. For context, Forto runs monthly one-day hackathons where engineers are encouraged to work on individual projects not related to their day-to-day work. The projects can range from machine learning to music synthesis. Still, sometimes we use those hackathons to prototype or implement some solution that is related to our work but somehow isn’t prioritized for the time being.
The versioning tool of choice for this first hack day was Version. While our Versio implementation worked for incrementing version numbers and producing releases of our modules, we encountered a couple of papercuts, and as more engineers within the company started making module changes, the downsides of the added complexity became apparent. After a few months, we decided to take a second hack day to see if a simpler from-scratch solution could work more effectively.
Our new workflow still allows Terraform modules within a monorepo to be individually versioned. However, the release process is now tied closely to the Pull Request lifecycle. After a Pull Request is merged, a new version number is calculated and the produced artifacts are uploaded to an S3 bucket. Finally, the repository’s Wiki is updated with automatically generated module documentation along with a new changelog entry. This results in a simpler versioning workflow, while leveraging various Github features such as PR labels, Github Actions, and the GraphQL API.
In the following sections, we will explain why we version our modules, share the decisions we made and show an example flow of a module upgrade.
We also created a public reference repository github.com/freight-hub/terraform-modules-demo so anyone can easily implement this workflow on their own repositories.

(“Freighthub” is Forto’s previous brand name. Renaming our Github org is top on the Jira board, honestly!)

Make a change in some module

You know, your job 🙂

Picture change (diff) in some Terrafrom file
Adding a new variable

Test the new module before release

If you want to test your code locally, you can use file system path-based module source reference, like:

source = “../../../modules-repos/name_of_module”

If you want to have the unreleased change tested by the CI / CD pipeline (see previous article)you can use git-based module source reference, such as:

source = “[email protected]: freight-hub / terraform-modules-demo.git? ref = BRANCH-NAME ”

Open a PR for the change

Make sure to add the text intended for the changelog in the PR description. Providing a meaningful changelog text ensures that the release history will be accurate and useful.

PR open page, showing the text placeholder

Apply a release increment label to the PR

Based on Semantic Versioning guidelines.

Merging is blocked until this is done.

Picture of the PR label selector drop down, showing “major,” minor ”,“ patch ”and“ no-release ”
Choose the relevant label, appreciate the nice color scheme
The pipeline will post a PR comment including current & new version numbers.

Picture of the automated planned version comment
Just a minor release? You better hope the new variable has a nice, safe default value!

Github action job log showing a wiki update
Github wikis are implemented as Git repositories… Act surprised.
Github action job log showing the module zip being uploaded to s3
Uploading the module zip to s3

Here we see the new variable in the autogenerated wiki page:

Short snippet form the autoganerated wiki document, showing the new Terrafrom variable
The new variable is now documented, yey!
A picture of the autogenerated changelog
The variable is documented above, so I did not add any context, seems a bit short though

Private repos should use S3 URLs to allow Terraform to use AWS authentication and authorization, public repos can just use plain HTTP URLs

source = “s3:https://forto-terrform-modules-demo.s3.eu-west-1.amazonaws.com/terraform-aws-vpc-1.1.0.zip”

Why version your modules at all?

Deterministic behavior – surprises are good for birthdays (maybe), but not infrastructure as code repos! Operators should expect to see changes in their infrastructure objects only when they change the code in their Terraform workspace, regardless of work done on the modules. When a new version for a module is desired (eg fixing a bug, adding functionality), an explicit change must be made to the Terraform workspace, thus ensuring ownership policies like Github CODEOWNER Required Reviews are kept.

Why Semver?

Consumers of the modules can easily tell if they need to look at the changelog before upgrading as major versions indicate large or breaking changes. Note that Patch versions do not necessarily introduce less risk compared to minor or even major changes. Risk is managed by testing and code & plan reviews.

The CI / CD and permission / ownership models are totally different compared to the other Terraform repos. Having modules and workspaces root code in the same repo would result in a highly complex repo configuration.
# 2 Pull Request (PR) Based

We originally had a commit-based approach. Version would check all the [conventional] commit messages in a merged PR, decide on a version bump, and release it.

While we liked the pure git approach, it resulted in some confusing user experiences. For example, commit messages from a “squash” merge might pop back into existence and fail the Versio release command. This error happens after the merge is complete which makes the damage annoying to repair.

And because GitHub does not provide centralized pre-commit hooks, we could not easily enforce a policy on those commit messages, requiring users to carefully follow the repository instruction or be forced to re-write the PR’s git history.

For our from-scratch solution, we started leveraging the Github PR itself. A set of four PR labels (major, minor, patch, and no-release) replaced the complex scanning of commit messages for versioning markers such as “feat!” and “fix:”. With git history ignored, the version bumps became much more reliable and predictable.

It also allowed us to post the “planned” versioned in PR comment and disallow merges of PR with no label, ensuring attention was given to version bump size. This was implemented with our own code, in languages ​​which we know (Typescript / YAML / Bash) and with about 95% fewer lines of code compared to Versio’s Rust codebase.

# 3 Keep the version “state” in Git tags.

The big downside of strict versioning is version drift. Upgrades must be done explicitly, and if the upgrades aren’t done rigorously, previous versions of the modules will be kept in use long after new versions become available.We plan to introduce another job for the Terraform modules pipeline to automatically open PRs which upgrade all module references in the workspaces root repo to the latest version. For minor and patch version bumps, these PRs should be ready to merge as-is. Major version bumps would instead open draft PRs, as we assume the large or breaking changes will require more work before the PR is ready to merge.

I hope you found this article relevant! Please checkout (pun very much intended) our reference repo: https://github.com/freight-hub/terraform-modules-demo.It should be possible to clone it and adjust the CI to your needs in a few hours. Finally, do not forget to visit our careers page! We are doing all kinds of other cool stuff and most of it does not end up in the blog.

Source

Latest news
Related news