Cloud

Picture this: You’re ordering an Uber in Lisbon, but your request takes a scenic tour through Madrid, London, and Virginia before confirming your ride. That was the case for millions of users until Uber and Google Cloud set out on an even bigger journey: redesigning how global edge networks should work.

Operating across six continents, Uber connects millions of riders and drivers while handling more than 100,000 concurrent trips and more than a million HTTP requests per second.

At this scale, every millisecond matters. When Uber’s existing edge architecture had sub-optimal routing paths, the company partnered with Google Cloud to redesign their global network approach. The results: substantial latency improvements and millions of dollars in cost savings.

The challenge: Sub-optimal routing, inefficient architecture with operational overhead

Uber’s previous Google Cloud edge used the open source Envoy proxy instances running on virtual machines across 16 regions. While designed to reduce latency by bringing services closer to users, this architecture often created sub-optimal routing paths with multiple unnecessary hops through different regions before reaching Uber’s data centers. The additional network transit increased latency and degraded the user experience that Uber’s customers expect.

This setup presented several challenges:

• Operational complexity: Managing and orchestrating a large fleet of virtual machines (VM) was cumbersome and deviated from Uber’s internal standards.
• Diminishing returns on latency: Contrary to initial assumptions, running Envoy in numerous global regions did not consistently improve latency for all users. In fact, for some, it introduced unnecessary network hops.
• High operational costs: Maintaining a large, globally distributed infrastructure incurred significant costs.

The solution: Direct routing with Hybrid NEGs

The goal was straightforward: create the most direct path from the user to Uber’s backend services across on-premises and various cloud environments. The approach involved moving away from the distributed Envoy VMs and using Google Cloud’s Hybrid Network Endpoint Groups (NEGs) instead.

This new architecture, developed through a 10-month collaboration between Uber and Google engineers, directs traffic from Google’s Global External Application Load Balancer — fronted by Google Cloud Armor for DDoS protection and Cloud CDN for caching — directly to Uber’s on-premises infrastructure via Cloud Interconnect.

The results of migrating to Hybrid NEG-based load balancers were immediate. By removing all edge VMs, the traffic path became significantly more efficient, allowing Google’s global network to handle the long-haul transit over optimized channels. This shift delivered a 2.6% latency improvement at the 50th percentile and 10% at the 99th percentile, directly improving service responsiveness.

The results: impactful improvement

The migration delivered substantial improvements across three key areas. After validating the design and shifting 99% of edge traffic, the project achieved:

• Significant cost reduction: Removing the entire fleet of edge Envoy VMs resulted in a significant cost savings.
• Improved performance and user experience: The streamlined traffic flow improved latency for Uber’s mobile app users by 2.6% at p50 and 10% at p99.
• Simplified operations: Decommissioning the edge VMs reduced operational overhead and improved reliability through more standardized tooling.

“At Uber, every millisecond defines the user experience for millions of people. By re-architecting our global edge with Google Cloud and Hybrid NEGs, we’ve created a more direct, lower-latency path for our services. This not only enhances today’s user experience but also provides the high-performance foundation necessary for our next generation of AI applications, all while significantly reducing operational overhead for our engineering teams.” – Harry Liu, Director of Networking, Uber.

Key takeaways for enterprise teams

Uber’s edge architecture transformation demonstrates what focused technical collaboration can achieve. By replacing a distributed Envoy VM fleet with a streamlined architecture using Google’s global network and Hybrid NEGs, Uber achieved significant improvements in performance, cost, and reliability.

This migration succeeded in under a year through close collaboration between Uber and Google engineers. Key success factors included:

• Architectural validation: Google’s insights into its load balancer architecture helped validate that fewer proxy locations would improve performance, reduce operational overhead.
• Performance modeling: Google engineers modeled production-scale results from Uber’s initial tests, saving benchmarking time and providing the confidence to proceed.
• Simplified design: Hybrid NEGs eliminated the need for Envoy proxy VMs in Google edge.

Multi-tenant GKE platform characteristics

As outlined in Google’s public documentation, there are multiple considerations that need to be addressed when running and operating a GKE cluster. The platform team worked diligently to address the following considerations and satisfy various tenant team’s requirements:

• Workload placement: The platform team assigns labels to node pools to support workload affinity, and tenants use a combination of taints and tolerations to ensure workloads are scheduled on the right node pools. This is necessary as each node pool has distinct firewall requirements based on the type of traffic it handles (HTTPS, POPS, etc). Additionally, Resource Manager Tags are used to govern firewall rules and associate the node pool with the applicable firewall rule.
• Access control: Kubernetes Role-based Access Control (RBAC) is the primary mode of governing and restricting user access to cluster resources. Each tenant has one or more namespaces within their cluster and the cluster is bootstrapped with standard policies during the tenant onboarding process.
• Network policies: All clusters are provisioned under dataplane v2, and Yahoo Mail uses the standard Kubernetes Network Policy to control traffic flow. Under the hood, this uses Cilium, an open-source solution for providing, securing, and observing network connectivity between workloads.
• Resource quotas: To optimize resource utilization and prevent overconsumption, the platform team enforces resource quotas for cpu/memory within a namespace.
• Scaling: This is determined by the platform team based on the tenant’s quota requests during onboarding. Due to certain feature limitations associated with node auto-provisioning around usage of secure tags for firewall rules and defining a specific machine type and shapes, it could not be utilized, but we worked with the engineering team to develop feature requests to address this gap.

Challenges

Through the course of this migration, both Yahoo Mail and Google faced technical constraints and challenges. Below, we outline some of the key challenges faced and the approaches taken to address them:

• Connecting to the control plane: As with most enterprise customers, Yahoo Mail provisioned private GKE clusters and needed connectivity between the control plane and its CI/CD tool (screwdriver) from outside the VPC network. The platform team deployed bastion hosts that were used to proxy the connection to the control plane, but it faced scalability challenges. Google worked with the customer to test out two solutions using Connect gateway and DNS-based endpoint to obviate the need for a bastion host.
• End-to-end mTLS: One of Yahoo Mail key security tenets was to ensure end-to-end mTLS, which the architecture’s overall design and underlying Google Cloud services should be able to accommodate. This resulted in significant problems as one of the key load balancing products (Application Load Balancer) did not offer end-to-end mTLS at the time. We explored alternative measures such as implementing a bespoke proxy application and using Layer 4 load balancers throughout the stack. Professional services also worked with Google Cloud engineering to define the requirements for mTLS support as part of Application Load Balancer.
• Integration with Athenz: Yahoo Mail used an internal tool for identity/access management, Athenz that all Google Cloud services needed to integrate with to perform workload identity federation. Within the Kubernetes context, this meant that users still needed to be authenticated via Athenz, but use workload identity federation as a mediator. As workload identity federation was also a fairly new feature, we needed to collaborate closely with Google Cloud engineering to implement successfully in the Yahoo Mail environment.
• Kubernetes externalTrafficPolicy: One of the distinct features that Yahoo Mail had been most vocal and excited for was weighted routing for load balancers. This feature would allow for optimal routing of incoming traffic to the backends. While this was supported for managed instance groups, there was no native integration with GKE at the time. In its absence, the platform team had to explore and experiment with externalTrafficPolicy modes such as local and cluster mode to determine its performance impact/limitations.
• Capacity planning: Last but not least, Google Professional Services performed capacity planning, a cornerstone of a successful cloud migration. Here, it entailed collaborating across multiple application teams to establish a baseline for capacity needs, making key assumptions, and estimating the resources required to meet both current and future demands. Capacity planning is a highly iterative activity that needs to evolve as workloads and requirements change. Thus, conducting regular reviews and maintaining clear communication with Google was paramount to ensure that the cloud provider could adapt to the customer’s needs.

Yahoo Mail’s next milestone

Migrating an application as big as the Yahoo Mail application is a huge endeavor. With its two-pronged approach — lift and shift for most services, and strategic rearchitecting for some — Yahoo is well on its way to setting its mail system up for the next generation of customers. While a small portion of the Yahoo Mail system is in production, the majority of its services are expected to be onboarded over the next year. For more information on how Google PSO can assist with other similar services, please refer to this page.

We would like to express our gratitude to the Yahoo Mail Compute Infra team for their cooperation in sharing details and collaborating with us on this blog post.

In today’s fast-paced cloud-native world, the speed at which your applications can start and scale is paramount. Faster pod startup times mean quicker responses to user demand, more efficient resource utilization, and a more agile development and deployment lifecycle overall. We’re continuously working to enhance the performance of Google Kubernetes Engine (GKE) to help you achieve these goals.

Previously we introduced container image streaming in GKE, a feature designed to significantly reduce image pull times and accelerate application startup. Today, we’re excited to announce a new set of performance improvements to GKE container image streaming.

These enhancements can help your GKE workloads start up faster and run more efficiently, particularly ones suffering from long startup times due to large container images. Specifically, AI/ML model serving applications will benefit from the improved startup times.

What’s new?

The performance boosts stem from a combination of targeted client-side innovations and ongoing optimizations to our image-streaming backend infrastructure.

A key improvement on the client-side is new intelligent read-ahead capabilities. These allow GKE to proactively fetch image data that is likely to be requested next, minimizing the time your applications spend waiting for data during startup. This works in concert with improvements to the image streaming backend, ensuring that your containers get the data they need, when they need it — just faster.

Alongside these client-side enhancements, we’ve made a number of improvements to our backend that help ensure that the image data is served efficiently and reliably, contributing to the overall speed and stability of the image streaming process.

Measuring the gains: Our benchmarking approach

We benchmarked the performance of image streaming to quantify its benefits, using an internal benchmarking suite to compare historical image streaming data versus the latest version. The benchmark is based on the popular Triton Inference Server image, which we use to measure image loading performance.

In general, we can see a up to ~30% improvement on the image streaming performance with the newly added enhancements. For large container images such as AI/ML models (e.g., vLLM-based containers, which often start at 8 GB and that can easily be as big as 100GB when you include model weights), this enhancement makes a big difference. By improving image pulling time, you get overall quicker container startup compared to non-streamed containers. These performance enhancements also play a big role in scaling out containers using horizontal pod autoscaler (HPA).

Get started with faster image streaming today

These image streaming performance improvements are automatically available to you when using GKE versions 1.32.1-gke.1729000 or newer. If you are already using container image streaming, there are no configuration changes you need to make to benefit from these enhancements. If you are not yet using image streaming, simply enable it on any new or existing GKE cluster to instantly get these benefits.

Improved image streaming performance marks another milestone on our journey to provide you with the fastest and most efficient container management platform. We will roll out further improvements to image streaming focused on image availability, usability, reliability, and integration with other GKE capabilities.

We encourage you to leverage these new enhancements to accelerate your application deployments on GKE. Check out the image streaming documentation to get started today and tap into a world of applications that start and scale faster!

What guides your approach to software development? In our roles at Google, we’re constantly working to build better software, faster. Within Google, our Developer Platform team and Google Cloud have a strategic partnership and a shared strategy: together, we take our internal capabilities and engineering tools and package them up for Google Cloud customers.

At the heart of this is understanding the many ways that software teams, big and small, need to balance efficiency, quality, and cost, all while delivering value. In our recent talk at PlatformCon 2025, we shared key parts of our platform strategy, which we call “shift down.”

Shift down is an approach that advocates for embedding decisions and responsibilities into underlying internal developer platforms (IDPs), thereby reducing the operational burden on developers. This contrasts with the DevOps trend of “shift left,” which pushes more effort earlier into the development cycle, a method that is proving difficult at scale due to the sheer volume and rate of change in requirements. Our shift down strategy helps us maximize value with existing resources so businesses can achieve high innovation velocity with acceptable quality, acceptable risk, and sustainable costs across a diverse range of business models. In the talk, we share learnings that have been really helpful to us in our software and platform engineering journey:

1. Work backwards from the business model: By starting with the business model, organizations can intentionally guide platform evolution and investment to align with desired margins, risk tolerance, and quality requirements. At Google, our central platform must support diverse business models, necessitating continuous strategic refinement and adaptation.
2. Focus on quality attributes for central software control: Quality attributes, such as reliability, security, efficiency, and performance, are emergent properties of software systems and are important for creating business value and managing risk. These are often referred to as “non-functional requirements” because they define how our software behaves, not what it functionally does. With a shift down strategy, we can embed the responsibility for assuring quality attributes directly into the underlying platform systems and infrastructure, thereby significantly reducing the operational burden on individual developers.
3. Abstractions and coupling are key technical tools to gain control of quality attributes: We define two key technical components in the way we build platforms: abstractions and coupling. In a shift down strategy, abstractions provide understandability, risk management levers, accountability, and cost control by encapsulating complexity. Coupling refers to the interconnectedness and interdependence of components within a system or development ecosystem. For a successful shift down strategy, the right degree of coupling is crucial because it allows the development platform and ecosystem design to directly implement and influence quality attributes. In fact, coupling is how we offer entire infrastructure and platform solutions as coherent services like Google Kubernetes Engine (GKE).
4. Shared responsibility, education, and policy are equally important social tools: Shared responsibility is a crucial social tool within software at scale. This is actively cultivated through education, such as training engineers on platform and AI usage, and fostering a “one team” culture that encourages a shift from artifact-bound identities to overarching mission goals and client-focused engagement. Furthermore, explicit policies like centrally enforced style guides and secure-by-design APIs are fundamental for embedding quality attribute assurance directly into the platform and infrastructure, significantly reducing the operational burden on individual developers by ensuring consistency and automated controls at scale.
5. Use a map. Supporting many business units with one platform is a vast and complex problem; we need a map. The ecosystem model is a framework that categorizes different types of software development environments, ranging from highly flexible, developer-controlled systems to highly opinionated, vertically integrated ones where the ecosystem itself assures quality attributes. Its critical purpose is to provide a visual and conceptual tool for evaluating how well our ecosystem controls match our business risk. This helps us ensure that the level of oversight and assurance of quality attributes aligns with the potential cost of mistakes. The goal is to be in the “ecosystem effectiveness zone,” where controls are balanced to mitigate significant risks from human error without imposing overly restrictive systems that negatively impact velocity and developer satisfaction.

6. Divide up the problem space by identifying different platform and ecosystem types.

Because the developer experience and platform infrastructure change with scale and degree of shifting down, it’s not enough to just know where the ecosystem effectiveness zone is — you have to identify the ecosystem by type. We differentiate ecosystem types by the degree of oversight and assurance for quality attributes. As an ecosystem becomes more vertically integrated, such as Google’s highly optimized “Assured” (Type 4) ecosystem, the platform itself assumes increasing responsibility for vital quality attributes, allowing specialists like site reliability engineers (SRE) and security teams to have full ownership in taking action through large-scale observability and embedded capabilities. Conversely, in less uniform “YOLO,” “AdHoc,” or “Guided” (Type 0-2) ecosystems, developers have more responsibility for assuring these attributes, while central specialist teams have less direct control and enforcement mechanisms are less pervasive. It’s really important to note here that this is not a maturity model — the best ecosystem and platform type is the one that best fits your business need (see point #1 above!).

Intentional choices in platform engineering

The most important takeaway is to make active choices. Tailor platform engineering for each business unit and application to achieve the best outcomes. Place critical emphasis on identifying and solving stable sub-problems in reliable, reusable ways across various business problems. This approach directly underpins our “shift down” strategy, moving toward composable platforms that embed decisions and responsibilities for software quality directly into the underlying platform infrastructure, thereby improving our ability to maximize business value with the right resources, at the right quality level, and with sustainable costs.

Watch our full discussion for more insights on effective platform engineering.

As the Hawaiian Islands’ primary higher education resource, the University of Hawaii (UH) System faces a unique challenge among U.S. universities: many graduates–including those with deep family roots in Hawaii – are soon confronted with a competitive job market, and struggle to land a fulfilling entry-level role and starting salary commensurate with Hawaii’s higher-than-average cost of living. This risks a continued outflow of skilled young grads from the Islands as they explore other career options on the mainland.

To bolster its mission of career readiness and workforce development, the UH System collaborated with Google Public Sector to rapidly prototype a new platform for Hawaii Career Pathways. Leveraging Google Cloud’s unified AI platform Vertex AI and BigQuery to analyze vast labor market data, this innovative AI-powered tool evaluates a student’s qualifications–including prior experience, current skills, education and certifications–along with their personal interests and career aspirations. The output is a customized student profile, which, with additional personalized guidance from Gemini, aligns their academic paths with tangible career opportunities within Hawaii. This human-centered approach empowers students to define their career possibilities along with key factors toward securing local jobs, charting a course for them to begin their careers in the state. It’s about fostering talent retention by enhancing student productivity and creating a direct bridge for Hawaii employers to connect with a prepared, local talent pool, directly preparing students for the demands of the greater market and Hawaii’s evolving economy.

Embracing the future: AI integration for education and beyond

In today’s increasingly AI-driven world, preparing students for the workforce requires more than just traditional curricula. UH is proactively integrating artificial intelligence across its educational curriculum and campus operations. Powered by Google Cloud’s Vertex AI Model Garden, with 200+ foundation models, including the latest Gemini models, this integration enhances learning experiences, streamlines administrative tasks, and fosters innovation. As part of this commitment, UH launched Google AI Essentials, a free course for students and employees that teaches foundational knowledge, prompting techniques, and responsible AI practices. This comprehensive AI integration ensures students are not only learning about AI but are also using AI-powered tools, preparing them for jobs of today and tomorrow.

Ensuring digital access across a unique Island culture

A core tenet of the UH community is ensuring accessibility across the Islands’ broad cultural makeup. Google’s commitment to digital inclusion in Hawaii and the Pacific, highlighted by the Grow with Google partnership, strongly aligns with UH’s mission. Accommodating the unique demographics of its student body, UH has begun to leverage Google Translate to communicate to Pacific Islander students in their native languages, including Hawaiian, Māori, Samoan, Tongan, Cook Islands Māori, Cantonese and Marshallese. This directly benefits the diverse population of Hawaii, making communication more accessible. For the UH System, with its extensive engagement with U.S. territories and Pacific Island nations, this enhancement facilitates better communication, supports cultural preservation efforts, and bolsters academic and outreach programs.

Building upon a history of university innovation

This latest collaboration marks the latest chapter of Google’s integral relationship with the UH System. As a longtime Google for Education customer, UH Information Technology Services (ITS) was among the first universities in the U.S. to adopt Google Workspace for Education in the late 2000s. Today, Google Workspace for Education is used by more than 80 of the top 100 universities, as ranked by US News & World Report 2025, making it a leading collaboration and productivity suite for students, faculty, and staff.

Following this initial success, our partnership with UH expanded to include intelligence-driven security with Google Security Operations to deliver a full-stack IT platform. Additionally, Google Cloud plays a central role in UH’s AI-optimized High Performance Computing (HPC) infrastructure, enabling supercomputing-class performance and AI-intensive workloads, to maintain UH’s ranking among world class research universities.

By further equipping students with leading-edge tools like Google Workspace, Gemini, Vertex AI Platform, Google Security Operations, BigQuery, Google Translate and resources for academic achievement, UH ensures they have the opportunity to build successful lives and careers in their island home, strengthening the state’s community and economy for generations to come.

Learn more from Google Public Sector

Join us to learn how public sector organizations are leveraging Google Cloud’s AI and security solutions to achieve their goals at the Google Public Sector Summit 2025 on October 29. Sign up to receive our Google Public Sector newsletter for the latest updates, customer stories, announcements, and more.