GCP – Predictable serverless deployments with Terraform

As a software developer, you want your code to reliably work. If your code is deployed in any sort of complex architecture, your code may be correct, but a misconfigured deployment could mean the entire system doesn’t work. The ability to reliably deploy complex infrastructure is essential. Having detailed documentation is useful, but just one misconfiguration can cause many issues. 

In these cases, consider Infrastructure as Code as a way to achieve reliably, repeatable deployments. One tool that’s widely used right now is Terraform, which supports many major cloud platforms, including Google Cloud. 

Here’s a short example of how Terraform can help: consider the following gcloud command: 

This command will create a new storage bucket for you, but if you run it again, you get an error message that the bucket already exists! You could add manual checks around this command to ask if the bucket already exists, and create it if it doesn’t, but when you start adding these checks around all your scripts, it gets complex and unmaintainable.

Replacing fallible shell with reliable Terraform

With Terraform, you describe your desired state—in this case, a bucket exists in your project—and Terraform will take the steps required to make sure that state is met. 

If the bucket already exists, Terraform will take no action. But if the bucket doesn’t exist, Terraform will take the steps required to create it.

You write Terraform manifests in HCL—Hashicorp Configuration Language—which allows for such complexity like variables and calculated fields. 

Terraform will also work out the dependency graph itself, when working with multiple resources. Some resources have to be created before others. And some resources create data that will be used by other resources. For example, if you have a Cloud Run service that relies on that cloud storage bucket, the bucket has to exist first; and Terraform will work that out. If you have, say, a deployment of 5 cloud functions that are independent of each other, terraform will run those creations in parallel; which will be much faster than creating each one of those one by one. 

This technology also isn’t limited to just serverless products. With the Google Terraform provider you can deploy virtual machines, networking, and other complex infrastructure that would be downright annoying and frustrating to have to manually deploy over and over again.

If you want to use Terraform to provision a development environment, consider what should be different compared to your production setup. You may want to add some variables to say, create a smaller Cloud SQL instance rather than a production-spec one, but with Terraform you can easily create duplicate setups. 

Be aware of the limitations

Infrastructure as Code is good for infrastructure and deploying existing assets, like containers and compiled code. There are other tools that you use to build your containers, and Terraform is not one of those tools. Terraform can be the replacement for the manual deployment after your containers are built, in that existing setup. Alternatively,  it can be integrated into your existing automation, for example as a step in your Cloud Build configuration.  This works well if you are doing in-house development with continuous deployments. Check out “Managing infrastructure as code with Terraform, Cloud Build, and GitOps” for an example of how to implement this configuration. 

For complex deployments, consider using Terraform. Not only will your deployments get more reliable, you can store your live infrastructure configuration settings along with your code in source control.

Terraform in practice

For an example deployment, follow Katie and Martin as they deploy a sample cat identification service, with Terraform: 

The application demonstrated in this video and post is available on GitHub in the Serverless-Expeditions repo under the terraform-serverless folder. 

In this video, we deploy a Cloud Function and Cloud Run service, together with a Cloud Storage bucket, and various IAM configurations, to get the project up and running swiftly. We then look around the Google Cloud project to see what was created, and try making some changes that are then re-asserted by Terraform. Finally, we destroy all the Terraform created resources, and re-create them again, restoring the application. 

Learn more: 

• https://cloud.google.com/solutions/managing-infrastructure-as-code

• https://registry.terraform.io/providers/hashicorp/google/latest/docs 

Check out more Serverless Expeditions 

Serverless Expeditions is a fun and cheeky video series that looks at what serverless means and how to build serverless apps with Google Cloud. 

Follow these hosts on Twitter at @glasnt and @martinomander.