It's where we're going to start. Pre-commit v2 release poped in early 2020. It's a fantastic tool to use in combination with your IDE OnSave formatting (if you have some) so that you make sure to always push clean commits (depending on the rules you've set up).
First, we'll see how to customize your workspace for you to be able to launch pre-commit scripts when you run the git commit command.
Python (python3 preferred) and pip (pip3 😉), and that's all.
apt update && apt install -y python3 python3-pip
pip3 install pre-commit --upgradeAn example of .pre-commit-config.yaml
# .pre-commit-config.yaml
default_stages: [commit]
repos:
- repo: https://github.com/commitizen-tools/commitizen
rev: v2.18.0
hooks:
- id: commitizen
stages:
- commit-msg
- repo: https://github.com/antonbabenko/pre-commit-terraform
rev: v1.50.0
hooks:
- id: terraform_fmt
- id: terraform_validate
- id: terraform_tflint
- id: terraform_tfsec
- id: checkov
- id: terraform_docs_replace
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.0.1
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-filesIn this key file, you'll choose hooks from a .pre-commit-hooks.yaml file defined in git repositories and default_stages on when to trigger them.
For this example, you may need to "install" a specific hook-type : commit-msg
pre-commit install --hook-type commit-msg
First, add some terraform code
# main.tf
terraform {
required_version = ">= 1.0.0, < 1.0.4"
}
resource "random_password" "password" {
length = 16
special = true
override_special = "_%@"
}
output "test" {
value = random_password.password.result
sensitive = true
}Manually trigger hooks to confirm that everything works pre-commit run -a
root@655088f99f35:/test# pre-commit run -a
[INFO] Initializing environment for https://github.com/commitizen-tools/commitizen.
[INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/antonbabenko/pre-commit-terraform.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Terraform fmt............................................................Failed
- hook id: terraform_fmt
- files were modified by this hook
main.tf
Terraform validate.......................................................Passed
Terraform validate with tflint...........................................Failed
- hook id: terraform_tflint
- exit code: 127
/root/.cache/pre-commit/reposnevwej6/terraform_tflint.sh: line 66: tflint: command not found
Terraform validate with tfsec............................................Failed
- hook id: terraform_tfsec
- exit code: 127
/root/.cache/pre-commit/reposnevwej6/terraform_tfsec.sh: line 25: tfsec: command not found
Checkov..................................................................Failed
- hook id: checkov
- exit code: 1
Executable `checkov` not found
Terraform docs (overwrite README.md).....................................Failed
- hook id: terraform_docs_replace
- exit code: 1
/bin/sh: 1: terraform-docs: not found
Command 'terraform-docs md ./ > .//README.md' returned non-zero exit status 127.
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check Yaml...............................................................Passed
Check for added large files..............................................PassedExciting things get displayed, and that's because you have to install some of the tools as packages.
Now we'll be focusing on the tool Terraform. What may be the valuable hooks that we can install to increase your infrastructure as code quality? You've got some hints from the output above or from the config file section 🙂. After we've set them up, I'll be explaining their usefulness.
Install them (brew is also available for linux)
brew install tfsec tflint checkov terraform-docsNow, you can rerun pre-commits and see them in action.
Terraform fmt............................................................Passed
Terraform validate.......................................................Passed
Terraform validate with tflint...........................................Passed
Terraform validate with tfsec............................................Passed
Checkov..................................................................Passed
Terraform docs (overwrite README.md).....................................Failed
- hook id: terraform_docs_replace
- files were modified by this hook
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check Yaml...............................................................Passed
Check for added large files..............................................PassedThe majority show Passed, which is OK, and Terraform docs failed to indicate that files were modified by this hook which means that it created the doc as expected but didn't stage it with git yet. No more error then! Let's dive into the details of each hook.
They're here for code quality readability improvements:
The terraform fmt command is used to rewrite Terraform configuration files to a canonical format and style. It is the built-in terraform command to format code towards Hashicorp standards.The terraform validate command validates the configuration files in a directory, referring only to the configuration and not accessing any remote services such as remote state, provider APIs, etc. It is the built-in terraform command to check the integrity of the code towards Hashicorp standards.TFLint is a framework and each feature is provided by plugins. It finds possible errors (like illegal instance types) for Major Cloud providers (AWS/Azure/GCP), warns about deprecated syntax or unused declarations, enforces best practices and naming conventions.
Now is a big part of this article: it's where terraform-specific pre-commit hooks improve your code quality drastically. Based on updated databases of best practices, the following 2 packages display warnings or errors in the providers' resources.
A static analysis security scanner for your Terraform codeCheckov scans cloud infrastructure configurations to find misconfigurations before they're deployed.As an example, let's take a look at the rule aws-kms-auto-rotate-keys from tfsec : it ensures that every resource "aws_kms_key" has the instruction enable_key_rotation = true
It is a best practice to enforce rotation on secrets but if for some reason you want to set key rotation to false, you can skip it using this instruction:
resource "aws_kms_key" "bad_example" {
enable_key_rotation = false #tfsec:ignore:AWS004
}You can do the same with checkov with #checkov:skip=CKV_AWS_20:The bucket is a public static content host
define a standard way of committing rules and communicating it (using the cli provided by commitizen). It's the one I recommend the most. It enforces the commit message to follow the standard of feat(scope): message. It is handy for generating changelogs and incrementing correctly the semantic versionGenerate Terraform modules documentation in various formats. This tool frees you from writing the objects you're creating with your code as it does it for you. It is customizable through a .terraform-docs.yml file.
I showed you the essentials, but you definitely should check by yourself which tool best suits your needs. It's an open world; go explore!
Pre-commit hooks are great locally, but you'll contribute to a codebase shared with many other DevOps. So what if they don't have pre-commit installed?
Well, the side effect will be that others' changes may block you if they didn't run the pre-commit scripts for a moment. For instance, when you'll pull updates that weren't lint from the repo and then try to commit again. As a workaround, you may want to check somewhere if commits are san. Running pre-commit tools in a CI is one solution.
You can set up pre-commit to run in gitlab-ci. My colleague and friend Stan made this simple with its Docker image and this usage example. Once set, the job will fail on any pre-commit hook failure like below.
We've seen how to set up pre-commit hooks on a Terraform project, which tools to install, and how to keep your repo checked even with new collaborators. I've witnessed a better global code quality over my different terraform repos since using pre-commit hooks.
As a DevOps, what may be your needs for code quality checking? What other tech or language do you need to hook (like Ansible or Python)? Do the benefits of CICD code quality outweigh the time it takes to run and the resources it uses? What do you think?