Testing Your Terraform Infrastructure Code With Python

Let’s cover an API use case with Terraform HCL & Python

Let’s cover an API use case with Terraform HCL & Python

Image by the author

Today, most of the infrastructure code is done through Terraform. It’s been there for quite a while, has a strong community, and it’s multicloud. However, things start to be tricky when it comes to testing Terraform code. While Terraform uses its own language (HCL), its backend is written in Golang.

A good pattern for Terraform modules tests is terratest, but as you may have guessed, you will need to write these in Golang. Here we’re going to showcase how we can use plain Python with a powerful yet simple library tftest on existing Terraform HCL code.

Tftest

Tftest is a small Python library from Google. It enables you to do Terraform actions (plan|deploy|destroy) programmatically and retrieve the execution plan, output variables, etc.

The power of tftest lies with the potential combo with pytest. Besides, Python has really good SDK support on the different cloud providers, which makes it nice for testing cloud infrastructure.

Case study

Our setup will involve a simple Cloud Run API (serverless container runtime), but you can apply the above method for any infrastructure you deploy!

Simple infrastructure test

In the first example, we will do a simple test to get our hands around tftest. Let’s try the following

• Create a planfixture

• Assert that the output name of our container image is what we expected

• Check the output variables are expected

The plan fixture simply looks at which terraform module/directory we need to look up (and perform the apply command) and make the output variable available as an object.

As you can see, you can easily retrieve any outputs and/or variables to perform any tests that you would like!

Advanced e2e infrastructure test

We are going now to do an e2e test that would do the following:

• Deploy the API on Cloud Run

• Get the URL of the deployed service

• Generate an Auth session ready for request

• Perform the request and assert the response

• Destroy the API

The first three points can be again put in a fixture to keep our testing function at the minimum and to be able to reuse this one for other requests.

This time, our fixture will do an apply and a destroy at the end of the test.

It will also generate an auth session based on the deployed URL to be allowed to perform a request. We will wrap up this auth session through a request_wrapper function.

Let’s put this fixture in conftest.py:

And here’s our test file that would perform the request and assert the response:

This file is kept at a minimum, and the fixture can be easily reused for other endpoints testing.

Caveat

This setup works great for any serverless components (like Cloud Run) that don’t have a long cold start. Some Cloud services can sometimes take up to 15–20 minutes to be ready, making it tedious to include them as part of a CI pipeline.

Isn’t there a better solution?

The setup presented here works great on an existing terraform codebase. However, if you are starting a new project, there are more appropriate solutions.

Terraform already has in beta Terraform CDK, which allows you to directly use Python (or any other programming languages supported) to declare your infrastructure, which makes testing much easier.

Pulumi is also a great candidate, and it’s more mature on the CDK side.

If you want to work with AWS only, you can also use AWS CDK, but you lose the benefits of investing knowledge in an IAC framework that isn’t vendor-locked.

In any way, it’s nice to finally be able to use a standard programming language to also manage infrastructure as you can directly leverage all the testing toolkits included!

Happy testing.

Link to the full demo GitHub repository here.

Want to Connect?
Follow me on YouTube or LinkedIn