Cloud

Controlling cloud costs is always top-of-mind for organizations. But how? It can be difficult to surface wasted resources and figure out how best to optimize them without sacrificing performance or availability.

Here on the Google Kubernetes Engine (GKE) team, we’re eager to help in any way we can. And after participating in some recent capacity planning exercises of our own, we couldn’t help but notice that there are some low-hanging fruit when it comes to optimization opportunities: According to our internal research, up to one in ten of clusters across the GKE fleet is idle at any given time. Further, among over-provisioned workloads, 40% of them have provisioned 30 times the resources they actually use — and 11% of workloads have provisioned over 100 times the needed resources. 

Thankfully, there are many best practices for reclaiming this wasted capacity, many of which we’ve built directly into GKE. Read on for four low-effort, high-reward ways to reclaim resources and optimize your GKE spend. As an added bonus, when you reclaim wasted resources and implement these techniques, you’re also minimizing management overhead — and reducing your carbon footprint!

Let’s dive in.

Strategy 1: Activate the ‘optimize-utilization’ profile in the GKE cluster autoscaler

Level: Easy 
Impact: Medium

Rightsizing vCPU and memory is one of the primary ways to save on GKE costs. The online UK-based, carpooling marketplace BlaBlaCar recently went through the process and saw their CPU utilization go from 25% to 53%. GKE provides several built-in tools to help you do the same. For example, GKE cluster autoscaler supports several mechanisms for automatically resizing a cluster’s node pools based on the demands on your workload, which you can apply by specifying an autoscaler profile. After analyzing our customers’ clusters across the GKE fleet, we estimate that activating the optimize-utilization profile can reduce unallocated vCPU and memory by 20% on average. For most clusters, it is a flip of a switch that has no impact on application behavior or performance, but that could significantly reduce the number of vCPUs and/or VMs that you need.

Steps you should take:

Learn about the optimize-utilization profile by watching this video guide.

Read the documentation to learn how to activate it.

Activate it on your existing (and new) clusters.

Strategy 2: Shut down idle clusters

Level: Easy 
Impact: Medium-High

Turn off what you’re not using. This is a no-brainer, yet, among the GKE clusters that users are running, we found one in ten clusters are potentially running idle at any given time. While there’s no one definition of an idle cluster, these clusters:

Aren’t running any pods

Are out of date and about to lose connectivity to the control-plane nodes

Haven’t had any API interaction and object changes for a long time

Have had no changes to pod count and are running at very low utilization levels

Shutting down an idle (or near-idle) cluster is easy:

First, review the “GKE Active/Idle clusters” sample dashboard in Cloud Monitoring and confirm which of your clusters are indeed idle or significantly underutilized.

For this, go to Cloud Monitoring -> Dashboard -> Sample Library and in Categories, select Google Kubernetes Engine. The “GKE Active/Idle clusters” are available in the sample list of the dashboard.

Once you confirm the cluster state, proceed to shutdown the cluster using your preferred method.

Strategy 3: Establish an optimization and rightsizing routine

Level: Easy 
Impact: Medium-High

Poorly sized clusters and workloads create a significant amount of waste. As we pointed out above, among over-provisioned workloads, our internal research found 40% of them have provisioned 30 times the resources they actually use — and 11% of workloads have provisioned over 100 times the needed resources. 

GKE has you covered, with multiple tools to help you rightsize your clusters:

Check GKE’s built-in cost insights to quickly locate the clusters and workloads that would make the most difference if rightsized. 

Then, use GKE’s new, built-in workload rightsizing capability to get guidance on how to vertically rightsize your deployments.

If you’re happy with GKE’s workload rightsizing suggestions, consider activating the vertical pod autoscaler to automate the process and reduce operational effort.

To learn more, check our best practices and video guide on GKE autoscaling.

Strategy 4: Move to GKE Autopilot

Level: Easy-Medium 
Impact: High

Sure, reducing the cost of your infrastructure is important, but reducing your operations costs may be even more valuable, allowing you to free up valuable engineering talent, while helping you benefit from a more stable and secure environment.

This is what you get with GKE Autopilot mode of operation. Because it’s priced per pod resource request rather than by provisioned infrastructure, GKE Autopilot can achieve instant cost savings for the simple reason that you won’t get charged for any unused infrastructure that you provision. GKE Autopilot also eliminates one of the more important sources of waste and effort: inefficient bin-packing. And because GKE Autopilot implements configuration best practices by default, you need less Kubernetes expertise to set up your environment correctly, so spend less time managing the system. 

On the basis of list price alone, GKE Autopilot might seem more expensive on a core/hour basis, but your reality might be different. When we looked across the entire GKE fleet, we found that as much as 45% of all clusters would be cheaper if they were migrated to GKE Autopilot today.

Want to get started with GKE Autopilot?

Get to knowGKE Autopilot, its benefits and limitations

Watch the intro video to GKE Autopilot

Create your Autopilot cluster

How low did your GKE costs go?

Here at Google Cloud, we’re passionate about making it easy for you to set up, run and manage your environment easily and cost-effectively. By implementing these four strategies today, we believe you will see a noticeable change in your GKE bill, as soon as next month! Even better, you’ll lessen the load on your GKE administrators — and you may even see a reduction in your carbon footprint. Share the results  of your GKE optimization efforts with us and the world on social media by using the hashtag #GKEoptimized.

Editor’s note: Sarah Masotti is a Digital Transformation Lead at Google Cloud, where she works with customers like Unilever, British Telecom, HSBC, and WPP. She excels at identifying innovation opportunities, and bringing them to life through Google’s products and services, and change management and transformation programmes, all with an innovative mindset.  

What is the coolest thing about your job?

I get to work with world class companies who are experts in their field, and think about how we can take their expertise, and combine it with Google’s to create something new. I help organizations identify their most pressing business challenges, and then figure out how to help solve them, tapping into all products, services, and programmes Google and our partners have to offer. 

I also really enjoy helping customers meet sustainability goals. Data centres alone are responsible for 1% of the world’s energy consumption, and by moving to the cloud, companies can shut down their data centers (and the energy used to power them), saving 60% – 85% in energy costs. And it’s especially cool when we partner with customers to innovate around sustainability. For example, we recently used Google technology to help end deforestation in Unilever’s supply chain, and it was pretty awesome to play a role in that and to see my customer present at COP26.

What’s the biggest challenge, and how do you address it?

Often customer requests go beyond cloud –  ‘what should my organization look like?’ ‘how can I innovate better’ ‘how should we fund initiatives’? The broad range of requests means I need to be really adaptable, and constantly learn and stay humble. I can’t be the expert in all of these things, and we have incredible talent and assets behind these topics, so a big part of my job is knowing when and how to leverage those best. 

What are the most effective things you do to help the transformation?

When you are asking people to change how they work, one of the most effective things you can do is listen. You need to understand and empathize before you can get people excited about a common goal to start to inflict change. Top-down executive sponsorship is essential, but you also want widespread education and local champions who are enthused about the new technology and  more agile ways of doing things, so they can help bring others along. Everybody tends to focus on the ‘new’, but I also like to look at what people will stop doing, such as manual tasks and siloed working. It is important to address those things too, or you will end up with technical debt. 

What in your past do you think helps you in your job?

Learning to work with different people in different cultures has helped me. I’ve worked in 25 countries, and have traveled to 40 more. Right out of university I was fortunate to get a job with PwC, one of Google Cloud’s earliest customers and partners. I had the travel bug from a young age, and raised my hand at PwC to work abroad, and they sent me to help transition 10,000 employees in Central Europe and Asia onto Google Workspace.  I learned how to adapt my message based on the audience, and how to create a network of employees to drive organizational wide change, and product adoption. I enjoyed working with small teams, to first enhance their proficiency so they could then take leadership roles to get others onboard. Pretty soon people were innovating on what they learned, improving how they handled job applicants and organized meetings and client projects. I loved working in new cultures and learning from different perspectives, and Google Cloud continues to enable those experiences for me, my teammates and our customers.

Back in July, the Kubernetes community announced the graduation of the Gateway API to beta, a big milestone for the group of people shaping the future of this networking API that defines the routing intent into a Kubernetes cluster, and even beyond, with service mesh capabilities through the GAMMA initiative. The Gateway API is the next generation of the Ingress API, with a broader scope and more use cases that can be addressed with it.

Today, we’re excited to announce the General Availability of the Google Kubernetes Engine (GKE) Gateway controller, Google Cloud’s implementation of the Gateway API, supporting single cluster deployments, in GKE 1.24+ clusters. 

A quick recap of the Gateway API

Platform operators are faced with the dual challenge of having to provide flexibility to developers consuming networking infrastructure while having to maintain control and consistency across teams.

Kubernetes is solving the need for the separation of duties between service owners and platform administrators with the Gateway API, which provides an open source, role-based standard for service networking. It is a structured way to describe network services consistently across a Kubernetes implementation, cross-tenants and cross-namespaces and be able to match the network resource with the appropriate roles. 

Previously each service owner made individual decisions about their choice of ingress, which could lead to inconsistencies in production due to the multitude of available solutions.

With the Gateway API, platform administrators can define policies and constraints on the networking infrastructure, and allow different teams or organizations to consume the shared networking services such as L4 and L7 load balancing while maintaining consistency and control.

The GKE Gateway controller

The GKE Gateway controller is Google Cloud’s implementation of the Gateway API for GKE clusters. It is a fully managed service with deep integration with GKE, enabling platform operators to manage internal and external HTTP(S) load balancing using an expressive and extensible API. Based on years of experience managing the GKE Ingress controller, we’ve engineered a controller that leverages the power of Google Cloud infrastructure and provides a highly redundant, scalable and multi-tenant service to manage load balancers for single- and multi-GKE cluster deployments, with centralized policy and control.

Advanced capabilities for platform administrators and service owners

Thanks to the flexibility and separation of concerns built into the Gateway API, GKE platform administrators can now attach a secure shared load balancer to a GKE cluster and let service owners define routing and traffic management policies to access their applications running in this multitenant GKE cluster.

By using the Gateway API with GKE, platform administrators and service owners get the following capabilities:

Shared Gateway for many routes/tenants – One of the pain points expressed by many customers regarding the Ingress implementation is the proliferation of load balancers as the cluster grows with more namespaces. By defining a single Gateway for a cluster, a single shared load balancer is created for the entire cluster, and each application team can define their own Routes to express the expected routing behavior for their namespace. Each Route is attached to a Gateway, which keeps the forwarding decision central without compromising the distributed nature of the Kubernetes configuration per namespace.

Global external and Regional Internal load balancing – The GKE Gateway controller comes by default with two GatewayClasses, one for the global external HTTP(S) load balancer (classic) (gke-l7-gxlb) and one for the regional internal HTTP(S) load balancer (gke-l7-rilb). Each GatewayClass can be used independently in the same GKE cluster to route traffic from the internet or from internal resources inside or connected to a VPC. 

End-to-end encryption at scale – The Gateway API defines how TLS can be implemented to provide a secure connection between the client and the gateway, as well as between the gateway and backend services. The GKE Gateway controller not only supports TLS for downstream and upstream traffic, but it does it at an unprecedented scale with an integration with Certificate Manager, to provide up to a million certificates for applications running in GKE. Google-managed SSL certificates or Kubernetes secrets can also be used as alternative options to secure connections between clients and the Gateway. As for upstream traffic (between the Gateway and backend services), TLS can also be used to ensure end-to-end encryption between clients and backend services.

Customized backend service properties – Many of our customers have particular needs when it comes to load balancing and especially how they want to maintain sessions with the backend services, or a custom health check. With a simple Policy attached to a Service, you can now define how session affinity or connection draining timeout should be handled, as well as custom health checks for applications that require a different port and/or path.

Advanced traffic management – The Google Cloud implementation of the Gateway API also gives you the ability to define how traffic should be handled from clients to applications. First of all, you can finally do something that was missing so much with Ingress: cross-namespace routing. But also, by leveraging the rich portfolio of features that the Google Cloud load balancers provide, you can gradually roll out new versions of services straight from GKE by using traffic splitting or canary deployments. Combined with backend service properties, this becomes a very flexible tool to migrate from version to version.

Let’s illustrate how the Kubernetes Gateway API simplifies traffic splitting across different services deployed in a cluster:

In this simple traffic splitting example, 90% of the traffic routed through the internal load balancer to store.example.com is directed to the store-v1 service and 10% of the traffic is sent to store-v2 service.

This can be achieved as simply as with the following manifest:

Looking ahead

Making the GKE Gateway controller generally available for single-cluster deployments is only a first step in our journey to create a real networking toolkit for platform administrators and service owners using GKE. Here are some other features that we’re working on. 

Multi-cluster Gateways

Within the next few months, we will continue our efforts to solidify our controller to ensure you can deploy a Gateway and Routes not just in a single cluster, but also in a multi-cluster environment (in Preview today). Using a multi-cluster Gateway across a fleet of GKE clusters will unleash new use cases including:

Multi-cluster or multi-region redundancy and failover

Low-latency serving through GKE clusters that have close proximity to clients

Blue-green traffic shifting between clusters (try this tutorial)

Expansion to multiple clusters because of organizational or security constraints

Multi-cluster Gateway, alongside multi-cluster Services, is a critical step forward for customers deploying global services who are seeking for more application availability without having to give up control over how the traffic is routed from one cluster to another.

New generation of Google Cloud load balancers

As mentioned earlier in this post, the GKE Gateway controller comes by default with two gateway classes — one for external load balancing and one for internal load balancing. Google Cloud recently announced the General Availability of the next generation of Global external HTTP(S) load balancers based on Envoy proxies. A new GatewayClass will soon be provided in the GKE Gateway controller to support this new load-balancing platform which provides advanced traffic management capabilities. We can’t wait to see what you build with these new load balancers within GKE.

Security 

When exposing services to the Internet, one of the first things that comes to mind is how to secure the platform and applications being exposed. Protecting the underlying infrastructure (the GKE nodes) is an obvious necessity that the GKE Gateway controller takes care of by managing the Cloud Firewall lifecycle for a given Gateway, but security is a multidimensional problem that must be tackled at different levels. In addition to the existing security features that the Gateway and the controller provide, we are continuing our effort to reinforce security for applications and will provide additional security features like SSL policies, HTTP redirects, Identity-Aware Proxy (IAP), and Cloud Armor.

Next steps

There’s a lot going on in the world of GKE Networking. At KubeCon North America, we presented 35+ sessions across the conference. Here are a few focused on Kubernetes Networking:

Where did all my IPs go? Presented by Cynthia Thomas, Product Manager, Google Cloud

Cilium Updates, News, Roadmaps, presented by Purvi Desai, Director of Engineering, Google Cloud and team 

SIG-Network: Introduction and DeepDive – presented by Rob Scott and Bowie Du, Google Engineering

Kubernetes Networking Infrastructure Offload – presented by Bloomberg, Intel, Nvidia, and Google

For more on Kubernetes Networking:

Visit GKE Network Recipes for Gateway Demos

Listen to the Podcast on Gateway Controller (from April 2022)

Try it free of charge

Google Cloud Marketplace is home to a wide variety of useful products for organizations across industries. The great selection it provides can create challenges, however, for cloud administrators who want to ensure that employees within their organization can easily view IT-vetted and approved applications. For example: when faced with such a broad range of products, employees might deploy an incompatible product version by accident; or they might not know your IT team prefers a specific product, leading them to adopt software your team hasn’t reviewed and approved.

To help avoid these issues and make it easier for your teams to quickly access the software and tools they need to do their jobs, we’re proud to introduce Private Marketplace, in preview today. The new Private Marketplace feature allows IT and cloud administrators to create a private, curated version of Google Cloud Marketplace that’s accessible to employees within their organization. With your own Private Marketplace, you can:

Curate product collections for your org: have your employees enter the Google Cloud Marketplace through a customized collection by default, surfacing the products that your IT team has selected first. This will ensure that they can identify pre-approved products quickly with certainty.

Prevent redundant products: When organizations — especially larger ones — use common products across teams and business units, it may be easier to negotiate deeper discounts or take advantage of volume savings promotions. Aligning on common products can also reduce complexity and simplify knowledge sharing.

Reduce “shadow IT”: When teams and business units buy products that your central IT or governance team doesn’t have visibility into, it can lead to non-compliant products and security risks. Setting up a Private Marketplace allows you to enforce controls by only showing approved products and encouraging their use. Improve visibility further by activating the Request Procurement workflow.

Best of all, setting up Private Marketplace is easy. And if you have teams or users that typically use different products from those offered in Google Cloud Marketplace, you can create multiple collections for each of them so they only see what’s most relevant.

How-to set up Private Marketplace

Private Marketplace is easy to set up with just a few simple steps. And while it isn’t a replacement for identity and access management (IAM) or organization policy, Private Marketplace can transform how your organization scales compliant product discovery:

Organization Administrators or Organization Governance Administrators can navigate to Google Cloud Marketplace > Marketplace Governance > Private Marketplace

Click Create collection.

Enter the name and brief description of the collection you are setting up.

In Add products, click + Add and paste the URLs of the product(s) you want to include.

Click Share and select the organization, folder(s), or project(s) you want to share this collection with and click Save.

Return to Marketplace Governance, and toggle the switch next to “Make your Private Marketplace visible in the Google Cloud Marketplace”

And that’s it. The users in the organization, folder(s), or project(s) you set will be able to access your Private Marketplace. And they’ll still have access to the wider Marketplace from there if they want to discover products that aren’t yet included in your collection. 

Learn more about this new preview feature set in the Create a Private Marketplace and Create and Share a Collection documentation.