GCP – Migrating Terraform resources to Cloud Run API v2

When making changes to Terraform code that requires no alteration to deployed resources, the Terraform state needs to be updated. This process requires removing Terraform state, updating the Terraform code, then re-importing Terraform state from the existing resources.

One of the times you might need to perform this operation is when there are new Terraform resources that are used to deploy and configure Google Cloud resources. This is the same with the newly released google_cloud_run_v2_service Terraform resource.

This blog post will describe this new Terraform resource, and describe the process of migrating an example Cloud Run service to this new resource.

Terraform resource differences

When Cloud Run was first launched, it was built on Knative and Kubernetes, and the v1 API reflected this model. There is now an additional v2 API that is built in a style similar to other Google Cloud APIs. Both APIs are generally available, and v2 does not obsolete v1.

v2 is being used by the Cloud Client Libraries, which includes the Google Provider in Terraform. Given the different shapes of these two APIs, google_cloud_run_service resources have different attributes to google_cloud_run_v2_service resources.

You may be familiar with having to declare metadata.annoations to configure settings around scaling and Cloud SQL connections. These settings are now defined as direct arguments. Additionally, within the template attribute, containers sat underneath, spec; spec in v2 is not required, and containers is a direct child of template. There are also some attributes that have had slight name changes.

Some of the metadata.annotations differences under template include (where the first is v1, and the second is v2):

“autoscaling.knative.dev/minScale” is scaling.min_instance_count”autoscaling.knative.dev/maxScale” is scaling.max_instance_count”run.googleapis.com/cpu-throttling” is containers.resources.cpu_idle”run.googleapis.com/cloudsql-instances” is volumes.cloud_sql_instance

Some of the attributes that have different names include:

spec.containers.timeout_seconds is containers.timeoutspec.containers.container_concurrency is containers.max_instance_request_concurrencyspec.service_account_name is service_accountspec.containers.env.value_from.secret_key_ref.key is containers.env.value_source.secret_ref_key.secretspec.containers.env.value_from.secret_key_ref.value is containers.env.value_source.secret_ref_key.version

Some of the attributes that have changed values include:

containers.resource.limits.cpuv1 example value: 1000mv2 equivalent: 1containers.timeoutv1 example value: 300v2 equivalent: 300s

A full list of attributes for both versions are available through the Terraform resource documentation: v1 google_cloud_run_service and v2 google_cloud_run_v2_service.

Updating Terraform code

Given the above differences, the changes required to use the new resource for an existing Cloud Run service may look like this:

In this example, we changed the integer value timeout_seconds to a string value timeout, changed container_concurrency to max_container_request_concurrency, and service_account to service_account_name. The spec level was also removed, and the resource itself was changed to google_cloud_run_v2_service.

Updating Terraform state

From a literal standpoint, the changes we’ve made to the Terraform code has no effective changes to the deployed Cloud Run service. However, because from a Terraform standpoint it’s being declared with a different resource, running terraform apply would destroy the Cloud Run service and recreate it. Since Cloud Run is stateless, this may not be a problem in most cases. However, there’s information about the history of the resource’s revisions you may want to keep, the timeline of revisions, etc.

To update the Terraform code and state without making any changes to the deployed resource:

start with the original code,confirm the service exists in your Terraform state, getting its resource name,using the resource names, get the resource identifiers,update the terraform code to use the new resource,remove the Terraform state for the resources,import state for the resources, andconfirm there are no pending changes in Terraform.

The rest of this post will go through each of these steps in detail.

In practice example

The example service we’ll use is a Cloud Run service running the sample Hello World image. This Terraform has already been applied to my project, and I’m keeping the state files locally (but I could just as easily store the state in a Cloud Storage bucket).

Original code

Confirm Terraform state

terraform state list

State is confirmed based on the configurations that are currently in the .tf files, so we’ll need to run this command before making changes.

From our configuration, we have two resources and one data source, so those are reflected in the state:

Get resource identifiers

echo RESOURCE.NAME.id | terraform console

For the later import step, the import command needs both Terraform’s resource name, but also the real world identifier of the resource.

From our example, the hello service’s Terraform resource name is google_cloud_run_service.default. We want the id argument on that resource, so we can use the console to view the value:

Now, this identifier is accurate, but it won’t work as is for our later step.

This identifier is in the form region/project/name, whereas according to the Cloud Run Admin API v2, it needs to be in the form project/region/name. We’ll come back to this later.

Update Terraform files

Now that we have information from our current state, we can make our configuration updates.

We’ll change the resource name (including associated references), remove the spec part of the template node (effectively moving container leftwards), and move the annotation to its new name.

We could make further changes in this step, but to ensure our terraform plan will notice no new changes, we’ll do that later.

Our updated file looks like this:

Remove old state

terraform state rm RESOURCE.NAME

We’re now ready to remove the old state. This will update the state files so Terraform will ‘forget’ about this resource. It won’t have any effect on the running resource, and is just going to remove our resource from the various .tfstate files we have.

Import new state

terraform import RESOURCE.NAME RESOURCE_IDENTIFIER

With the old state removed, we can now pull state from our project. Since we’ve defined the resource in our Terraform configuration, we want to use this resource name as our resource, and the identifier we adapted from the value earlier as the resource identifier.

If you didn’t get the format of the resource identifier right, you will end up with an error like Error 403: Permission denied on resource. Permission will be denied if the resource that’s being referenced doesn’t exist. Check the format of the resource identifier it parsed out, and adjust as needed.

Confirm no pending changes

terraform plan

Now we’ve updated the state, we can now confirm that Terraform can actively identify our new resource definition with our, still unchanged, real world resource.

By running terraform plan, we can confirm what Terraform would perform to our service to match the resource definition. If we did everything correctly, there would be no required changes.

We can run terraform apply to confirm that we’re in good standing with the consolidation of Terraform and real world state.

We have now successfully migrated a Cloud Run service defined in Terraform to a different resource, with zero changes to our deployed resource.

What we’ve learnt

In this article, we’ve learned about the v2 Cloud Run API, and how to adapt an existing deployed Cloud Run service to use google_cloud_run_v2_service. We also learnt how to remove and import Terraform state to reflect the real world state of our infrastructure, in a non-destructive manner.

Learn more

Terraform Resource definition for google_cloud_run_serviceTerraform Resource definition for google_cloud_run_v2_serviceTerraform Resource definition for IAM policies for Cloud Run