GCP – Automating FinOps cost management policies using Workload Manager
Do you find yourself battling surprise cloud bills? Do you spend more time tracking down un-tagged resources and chasing development teams than you do on strategic financial planning? In the fast-paced world of cloud, manual cost management is a losing game. It’s time-consuming, prone to errors, and often, by the time you’ve identified a cost anomaly, it’s too late to prevent the impact.
What if you could codify your financial governance policies and automate their enforcement across your entire Google Cloud organization? Enter Workload Manager (WLM), a powerful tool that lets you automate the validation of your cloud workloads against best practices for security and compliance, including your own custom-defined FinOps rules. Better yet, we recently slashed the cost of using Workload Manager by up to 95% for certain scenarios, letting you run large-scale scans more economically, including a small free tier to help you run small-scale tests. In this blog, we show you how to get started with automated financial governance policies in Workload Manager, so you can stop playing catch-up and start proactively managing your cloud spend.
The challenge with manual FinOps
Managing business-critical workloads in the cloud is complex. Staying on top of cost-control best practices is a significant and time-consuming effort. Manual reviews and audits can take weeks or even months to complete, by which time costs can spiral. This manual approach often leads to “configuration drift,” where systems deviate from your established cost management policies, making it difficult to detect and control spending.
Workload Manager helps you break free from these manual constraints by providing a framework for automated, continuous validation, helping FinOps teams to:
-
Improve standardization: Decouple team dependencies and drive consistent application of cost-control policies across the organization.
-
Enable ownership: Empower individual teams to build and manage their own detection rules for specific use cases, fostering a culture of financial accountability.
-
Simplify auditing: Easily run infrastructure checks across your entire organization and consolidate the findings into a single BigQuery dataset for streamlined reporting and analysis.
By codifying your FinOps policies, you can define them once and run continuous scans to detect violations across your entire cloud environment on a regular schedule.
Workload Manager makes this easy, providing you with out-of-the-box rules across Security, Cost, Reliability etc. Here are some examples of FinOps cost management policies that can be automated with Workload Manager:
-
Must have required label or tag for a specific google cloud resource (eg: BigQuery dataset)
-
Enforce lifecycle management or autoclass configuration for every cloud storage bucket
-
Ensure appropriate data retention is set for storage (eg: BigQuery tables)
-
Disable simultaneous multi-threading to optimize licensing costs (eg: SQL Server)
Figure – 1: Default Workload Manager policies as per Google Cloud best practices
Don’t find what you need? You can always build your own custom policies using examples in our Git repo.
Let’s take a closer look.
Automating FinOps policies: A step-by-step guide
Here’s how you can use Workload Manager to automate your cost management policies.
Step 1: Define your FinOps rules and create a new evaluation
First, you need to translate your cost management policies into a format that the Workload Manager can understand. The tool uses Open Policy Agent (OPA) Rego for defining custom rules. In this blog we will take a primary use case for FinOps — that is, to ensure resources are properly labeled for cost allocation and showback.
You can choose from hundreds of predefined rules authored by Google Cloud experts that cover FinOps, reliability, security, and operations according to the Google Cloud best practices or create and customize your own rules (checkout examples from the Google Cloud GitHub repository). In our example we will use one of the predefined ‘Google Cloud Best Practices’ rules for bigquery-missing-labels on a dataset. In this case, navigate to the Workload Manager section in your Google Cloud Console and start by creating a new evaluation.
Give your evaluation a name and select “Custom” as the workload type. This is where you can point Workload Manager to the Cloud Storage bucket that contains your custom FinOps rules if you’ve built one. The experience allows you to run both pre-defined and custom rule checks in one evaluation.
Figure 2 – Creating new evaluation rule
Step 2: Define the scope of your scan
Next, define the scope of your evaluation. You have the flexibility to scan your entire Google Cloud organization, specific folders, or individual projects. This allows you to apply broad cost-governance policies organization-wide, or create more targeted rules for specific teams or environments. You can also apply filters based on resource labels or names for more granular control. In this example, region selection lets you select where you want to process your data to meet data residency requirements.
Figure 3 – Selecting scope and location for your evaluation rule
Step 3: Schedule and notify
With FinOps, automation is key. You can schedule your evaluation to run at a specific cadence, from hourly to monthly. This helps ensure continuous monitoring and provides a historical record of your policy compliance. Optionally, but highly recommended for FinOps, you can configure the evaluation to save all results to a BigQuery dataset for historical analysis and reporting.
You can also set up notifications to alert the right teams when an issue is found. Channels include email, Slack, PagerDuty, and more, so that policy violations can be addressed promptly.
Figure 4 – Export, schedule and notify evaluation rules
Step 4: Run, review, and report
Once saved, the evaluation will run on your defined schedule, or you can trigger it on-demand. The results of each scan are stored, providing a historical view of your compliance posture
From the Workload Manager dashboard, you can see a summary of scanned resources, issues found, and trends over time. For deeper analysis, you can explore the violation data directly in the BigQuery dataset you configured earlier.
Figure – 5: Checkout evaluations for workload manager
Visualize findings with Looker Studio
To make the data accessible and actionable for all stakeholders, you can easily connect your BigQuery results to Looker Studio. Create interactive dashboards that visualize your FinOps policy violations, such as assets missing required labels or resources that don’t comply with cost-saving rules. This provides a clear, at-a-glance view of your cost governance status.
You can find Looker Studio template in template gallery and easily connect it with your datasets and modify as needed. Here is how you can use it:
-
Go to Looker studio.
-
Navigate to Templates and under Bigquery, select Google Cloud Workload Manager
-
Click on “Use your own Data” that asks for connecting the Bigquery table generated in previous steps.
-
After you have connected the Bigquery dataset, lick on Edit to create a customizable copy to incorporate any changes or share it with your team.
Figure – 6: Set up preconfigured Looker Studio dashboard for reporting
Take control of your cloud costs today
Stop the endless cycle of manual cloud cost management. With Workload Manager, you can embed your FinOps policies directly into your cloud environment, automate enforcement, and provide teams with the feedback they need to stay on budget.
Ready to get started? Explore the sample policies on GitHub and check out the official documentation to begin automating your FinOps framework today, and take advantage of Workload Manager’s new pricing.
Check out a quick overview video on how Workload Manager Evaluations helps you do a lot more across Security, Reliability and FinOps.
Then, review the updated pricing to learn more.
Read More for the details.
