gcp

As organizations race to deploy powerful GPU-accelerated workloads, they might overlook a foundational step: ensuring the integrity of the system from the very moment it turns on. 

Threat actors, however, have not overlooked this. They increasingly target the boot process with sophisticated malware like bootkits, which seize control before any traditional security software can load and grant them the highest level of privilege to steal data or corrupt your most valuable AI models.

Why it matters: The most foundational security measure for any server is verifying its integrity the moment it powers on. This process, known as Secure Boot, is designed to stop deep-level malware that can hijack a system before its primary defenses are even awake.

Secure Boot is part of Google Cloud’s Shielded VM offering, which allows you to verify the integrity of your Compute VM instances, including the VMs that handle your AI workloads. It’s the only major cloud offering of its kind that can track changes beyond initial boot out of the box and without requiring the use of separate tools or event-driven rules.

The bottom line: Organizations don’t have to sacrifice security for performance. There is a clear, repeatable process to sign your own GPU drivers, allowing you to lock down your infrastructure’s foundation without compromising your AI workloads. 

Our Secure Boot capability can be opted into at no additional charge, and now there’s a new, easier way to set it up for your GPU-accelerated machines.

Understanding the danger of bootkits

It’s important to secure your systems from boot-level threats. Bootkits target the boot process, the foundation of an operating system. By compromising the bootloader and other early-stage system components, a bootkit can gain kernel-level control before the operating system and its security measures load. Malware can then operate with the highest privileges, bypassing traditional security software.

This technique falls under the Persistence and Defense Evasion tactics in the MITRE ATT&CK framework. Bootkits are difficult to detect and remove due to their low-level operation. They hide by intercepting system calls and manipulating data, persisting across reboots, stealing data, installing malware, and disabling security features. 

Bootkits and rootkits pose a persistent, embedded threat, and have been observed as part of current threat actor trends from Google Threat Intelligence Group, the European Union Agency for Cybersecurity (ENISA), and the U.S. Cybersecurity and Infrastructure Security Agency (CISA). Google Cloud always works on improving the security of our solutions by strengthening our products and providing tools you can use yourself. In this article, we would like to demonstrate a new, easier way of setting up Secure Boot for your GPU-accelerated machines.

Limitations of Secure Boot with GPUs

Shielded VMs employ a TPM 2.0-compliant virtual Trusted Platform Module (vTPM) as their root of trust, protected by Google Cloud’s virtualization and isolation powered by Titan chips. While Secure Boot enforces signed software execution, Measured Boot logs boot component measurements to the vTPM for remote attestation and integrity verification. 

Limitations start when you want to use a kernel module that is not part of the official distribution of your operating system. That is especially problematic for AI workloads, which rely on GPUs whose drivers are usually not part of official distributions. If you want to manually install GPU drivers on a system with Secure Boot, the system will refuse to use them because they won’t be properly signed. 

How to use Secure Boot on GPU-accelerated machines 

There are two ways you can tell Google Cloud to trust your signature when it confirms the GPU driver validity with Secure Boot: with an automated script, or manually. 

The script that can help you prepare a Secure Boot compatible image is open-source and is available in our GitHub repository. Here’s how you can use it:

The script will execute each of the five steps described below for you. It may take up to 30 minutes, as the installation process takes this much time. We’ve also detailed how to use the building script in our documentation.

To manually tell Google Cloud to trust your signature, follow these five steps (also available in our documentation):

1. Generate your own certificate to be used for signing the driver.

2. Create a fresh VM with the OS of your choice (Secure Boot disabled, GPU not required).

3. Install and sign the GPU driver (and optionally CUDA toolkit).

4. Create a new Disk Image based on the machine with a self-signed driver, adding your certificate to the list of trusted certificates.

5. The new image can be now used with Secure Boot enabled VMs.

Whether you used the script or performed the task manually, you’ll want to verify that the process worked. 

Start a new GPU accelerated VM using the created image

To verify that everything worked, we can create a new VM using the new disk image with the following command (we enable the Secure Boot option to verify that our process worked).

Clean up

When you verify that the new image works, there’s no need to keep the verification VM around. You can delete it with:

Enabling Secure Boot

Now that you have built a Secure Boot compatible base image for your GPU-based workloads, remember to actually enable Secure Boot on your VM instances when you use those images! Secure Boot is disabled by default, so it needs to be explicitly enabled for Compute Engine instances.

When creating new instances

If you create a new instance using Cloud Console, the checkbox to enable Secure Boot can be found in the Security tab of the creation page, under the Shielded VM section.

For the gcloud enthusiasts, there’s –shielded-secure-boot flag available for the gcloud compute instances create command.

Updating existing instances

You can also enable Secure Boot for instances that already exist, however, make sure that they are running a compatible system. If the driver installed on those machines is not signed with a properly configured key, the driver will not be loaded. To update Secure Boot configuration for existing VMs, you’ll have to follow the stop, update and restart procedure, as described in this documentation page.

Get started 

Make sure to visit our documentation page to learn more about the process and follow our GitHub repository to stay up to date with other GPU automation news.

As developers and operators, you know that having access to the right information in the proper context is crucial for effective troubleshooting. This is why organizations invest a lot upfront curating monitoring resources across different business units: so information is easy to find and contextualize when needed.

Today we are reducing the need for this upfront investment with an out-of-the-box Application Monitoring experience for your organization on Google Cloud within Cloud Observability. 

Application Monitoring consists of a set of pre-curated dashboards with relevant metrics and logs mapped to a user-defined application in App Hub. It incorporates best practices pioneered by Google Site Reliability Engineers (SRE) to optimize manual troubleshooting and unlock AI-assisted troubleshooting.

Application Monitoring automatically labels and brings together key telemetry for your application into a centralized experience, making it easy to discover, filter and correlate trends. It also feeds application context into Gemini Cloud Assist Investigations, for AI-assisted troubleshooting.

1. Application, service and workload dashboards 

No more spending hours configuring application dashboards. 

From the moment you describe your application in App Hub, Application Monitoring starts to automatically build dashboards tailored to your environment. Each dashboard comprises relevant telemetry for your application and is searchable, filterable and ready for deep dives — no configuration required. 

The dashboards offer an overview of charts detailing the SRE Four Golden Signals: traffic, latency, error rate, and saturation. This provides a high-level view of application performance, integrating automatically collected system metrics across various services and workloads such as load balancers, Cloud Run, GKE workloads, MIGs, and databases. From this overview, you can then drill down into services or workloads with performance issues or active alerts to access detailed metrics and logs.

For example in the image below, a user defined an App Hub application called Cymbal BnB app, with multiple services and workloads. The flow below shows the automatically generated experience with golden signals, alerts and relevant logs.

2. Labels and context propagation 

See application labels propagated seamlessly across Google Cloud 

Once Application Monitoring is enabled, your application labels are propagated across Google Cloud, so you can see and use them to filter and focus on the most essential signals across the logs, metrics and trace explorers.

3. Gemini Cloud Assist Investigations

Troubleshoot issues faster with AI powered Investigations. 

Gemini Cloud Assist’s investigation feature makes it easier to troubleshoot issues because application boundaries and relationships have been propagated into the AI model, grounding it in context about your environment. 

Note – Gemini Cloud Assist Investigations is currently in private preview

Try Application Monitoring today

The new Application Monitoring experience provides a low-effort unified view of application and infrastructure performance for your troubleshooting needs.

Take advantage of the new Google Cloud Application Monitoring experience by:

1. Visiting your Cloud console

2. Setting up Applications in AppHub

1. Adding Services and Workloads to your Application

Related docs

1. Application Monitoring docs

2. AppHub docs

   1. Apphub coverage docs

For more than a decade, TPUs have powered Google’s most demanding AI training and serving workloads. And there is strong demand from customers for Cloud TPUs as well. When running advanced AI workloads, you need to be able to monitor and optimize the efficiency of your training and inference jobs, and swiftly diagnose performance bottlenecks, node degradation, and host interruptions. Ultimately, you need real-time optimization logic built into your training and inference pipelines so you can maximize the efficiency of your applications — whether you’re optimizing for ML Goodput, operational cost, or time-to-market. 

Today, we’re thrilled to introduce a new monitoring library for Google Cloud TPUs, a new set of observability and diagnostic tools that provide granular, integrated performance and accelerator utilization insights so you can continuously assess and improve the efficiency of your Cloud TPU workloads. 

Note: If you have shell access to the TPU VM and just need some diagnostic information (e.g., to observe memory usage for a running process), you can use tpu-info, a command-line tool for viewing TPU metrics.

Unlocking dynamic optimization: Key metrics in action

The monitoring library provides snapshot-mode access to a rich set of metrics, such as Tensor core utilization, high-bandwidth memory (HBM) usage, and buffer transfer latency. Metrics are sampled every second (1 Hz) for consistency. See the documentation for a full list of metrics and how you can use them.

You can use these metrics in your code directly to dynamically optimize for greater efficiency. For instance, if your duty_cycle_pct (a measure of utilization) is consistently low, you can programmatically adjust your data pipeline or increase batch size to better saturate the Tensor core. If hbm_capacity_usage approaches limits, your code could trigger a dynamic reduction in model size or activate memory-saving strategies to avoid out-of-memory errors. Similarly, hlo_exec_timing (how long operations are taking to execute on the accelerator) and hlo_queue_size (how many operations are waiting to be executed) can inform runtime adjustments to communication patterns or workload distribution based on observed bottlenecks.

Let’s see how to set up the library with a couple of realistic examples.

Getting started with the library

The TPU monitoring library is integrated within the LibTPU library. Here’s how to install it:

For JAX or PyTorch users, libTPU is included in your installation when you install jax[tpu] or torch_xla[tpu] (read more about PyTorch/XLA and JAX installation).

You can refer to the library in your Python code: from libtpu.sdk import tpumonitoring. You can then discover supported functionality with sdk.monitoring.help() and list available metric names using tpumonitoring.list_supported_metrics().

Example 1. Monitoring TPU duty cycle during training for dynamic adjustment 

Integrate duty_cycle_pct logging into your JAX training loop to track how busy the TPUs are.

A consistently low duty cycle suggests potential CPU bottlenecks or inefficient data loading. This example simply prints out the value, but in the real world you can trigger a re-sharding or other actions.

Example 2. Checking HBM utilization before JAX inference for resource management 

While running JAX programs on Cloud TPUs, optimizing HBM usage during compilation presents a significant opportunity. By proactively getting insights on potential TPU memory reservations during compilation, you can unlock greater efficiency and prevent out-of-Memory (OOM) errors, which is especially crucial for scaling large models. By checking the hbm_capacity_usage metric from the monitoring library you can see the available HBM, allowing for dynamic adjustments to your inference strategy and mitigating memory errors.

If HBM usage is unexpectedly high, you might consider optimizing your model size, batching strategy, or input data pipelines.

Maximize your TPU utilization

In this post, we showed you two simple examples of how you can improve the efficiency of your TPU workloads with some proactive monitoring. The TPU monitoring library can help you improve the utilization of your accelerator, dynamically tune them to your use cases, and ensure you have the best cost efficiencies. 

To learn more about the TPU monitoring library, please visit the documentation. To get started with Cloud TPUs, please visit our Intro to TPU documentation.

Partners have always been central to the Google Cloud ecosystem, becoming more and more instrumental in bringing Google’s AI innovations to enterprises. I am inspired by how partners have already built more than 1,000 agentic use cases across every domain to solve deeply entrenched pain points for our shared customers.

The emergence of agentic AI marks a true paradigm shift for technology, promising to reshape industries and redefine how businesses operate and create value. For our incredible ecosystem of partners, this represents a profound opportunity to lead customers into a more intelligent and automated future.

To chart this new territory, we’re releasing a new analysis today that provides a strategic framework for the journey ahead: Shaping the Future: The Transformative Potential of Agentic AI and the Strategic Imperative for Google Cloud Partners. 

An unprecedented opportunity for our partners

Our analysis¹ reveals that agentic AI is set to create a ~$1T global market for agentic AI services for partners. To put this in context, the projected $350B to $450B opportunity in the U.S. alone is larger than the entire U.S. traditional partner services market today.

Last year, our partner ecosystem influenced nearly 80% of Google Cloud’s incremental revenue growth. This year, IDC² expects our global system integrators to grow their Google Cloud AI practices as much as 100% as customers see increasing ROI. Now, with agentic AI, our partner ecosystem can build on this incredible momentum. Today, more than 90% of enterprises report interest in deploying agentic AI solutions within the next three years. Leading software players are also signaling their conviction, with agentic AI mentions in public filings 12x more today vs a year ago. The pace of adoption is electric and distinct value pools are fast emerging. Our ecosystem must act quickly to tap into the substantial, immediate opportunity already at stake.

The strategic leap for partners: From implementer to transformer

Leading in the agentic era means moving beyond technology implementation. To drive fundamental business transformation with agentic AI, SIs need to secure “quick wins” to deliver immediate ROI while simultaneously building toward “big bets” on long-term innovation. By providing value on high-impact workflows, partners can build the credibility and reusable IP needed to tackle the bespoke, multi-agent systems that will define the future. This journey is guided by six core principles:

1. Identify pain points and build prototypes: Focus on solving high-value, customer-specific pain points to drive differentiation, delivering working prototypes to prove value from day one.

2. Reimagine core business processes: Evolve the delivery to go beyond pure technical and systems execution, with initial focus on upfront consultative AI design to help customers restructure business processes by co-designing agentic workflows.

3. Use new tools to solve data gaps: Create opportunities to quickly overcome clients’ perceived data readiness challenges by using generative AI tools and modern inter-agent communication protocols to activate agents with less structured datasets.

4. Deploy agents at scale: Manage change and embed agents into business workflow, and establish processes to track agent performance. Bring robust change management capabilities to accelerate deployment and deliver clear ROI.

5. Manage the full agentic lifecycle: Deliver new forms of ongoing support, such as orchestrating agentic fleets, evaluating agent performance and refreshing agent knowledge. Create long-term competitive advantage by investing in reusable IP and reference integrations.

6. Innovate commercially: Consider different pricing models to align with the nature of agentic AI. Explore recurring, transaction-based, or outcome-based structures that reflect the measurable impact agents deliver.

Our commitment: Building the industry’s best agentic AI ecosystem with you

We are committed to supporting our partners at every level with a partner-first approach to services and delivery. We continue to infuse partners into every layer of our agentic AI stack to enable a truly open and thriving ecosystem, including:

• Open Innovation and thought leadership: We are committed to open-source contributions like the Agent Development Kit (ADK) and the Agent2Agent (A2A) protocol, fostering interoperability with support from over 100 industry leaders like ServiceNow and Workday. Our best-in-class infrastructure and pioneering AI research is powered by Google AI and DeepMind, who will continue to bring partners the leading technologies available.

• A purpose-built stack for agents: We provide a comprehensive toolkit to build, deploy, scale and commercialize agentic solutions with maximum interoperability. With best in class security and governance at all layers to ensure seamless and secure rollout of many agents across an enterprise. At its foundation, partners have choice across GPUs, TPUs and 200+ AI models. Partners can solve data readiness issues using BigQuery, develop custom agents on a unified platform with Vertex AI, commercialize them through the Google Cloud Agent Marketplace and make available to users on Agentspace.

• A thriving partner ecosystem: Our partner-first commitment extends beyond technology. We are increasing resources to help you address customer demand for AI agents, including a 2x increase in funding AI opportunities over the past year alone. We will continue to invest in your success with increased funding for critical partner training and enhanced co-selling programs to help you close larger deals, faster.

The future of AI will be shaped by those who lead from the front. The partners who act decisively now will not just participate in this evolution – they will define the category. I am thrilled about the journey ahead. I invite you to download the full analysis to explore these insights in greater detail and continue to share your perspectives on how best we can co-create the agentic future.

<sup>1.  Google Cloud commissioned Boston Consulting Group to analyze agentic AI TAM for services partners.
2. IDC InfoBrief, sponsored by Google Cloud, Google Cloud AI: Driving Opportunity and Growth for Global Consulting & Systems Integrator Partners, doc #US53276025, April 2025</sup>

Apache Spark is the cornerstone for large-scale data processing, model training, and inference for AI/ML workloads. Yet, the complexities of environment configuration, dependency management, and MLOps integration can slow you down. To accelerate your AI/ML journey, Dataproc now delivers powerful, ML-ready capabilities for Spark. Available on both Dataproc on Compute Engine clusters and Google Cloud Serverless for Apache Spark, these enhancements are engineered to streamline development and operations, reducing setup overhead and simplifying workflows. This allows data scientists and engineers to dedicate more time to building and deploying impactful models rather than wrestling with infrastructure.

Let’s explore what’s new and how to start using these innovations today.

AI/ML-capable runtimes

Getting a Spark environment ready for ML, especially with GPU acceleration, used to involve custom scripts and manual configuration. Dataproc now streamlines this with ML Runtimes. ML Runtimes is a specialized Dataproc on Compute Engine image version, starting from 2.3 for Ubuntu-based images, designed to accelerate ML workloads. It ships with pre-packaged GPU drivers (NVIDIA Driver, CUDA, cuDNN, NCCL) and common ML libraries such as PyTorch, XGBoost, tokenizers, transformers etc, significantly cutting down cluster provisioning and setup time. 

Google Cloud Serverless for Apache Spark also benefits from runtimes with pre-installed ML libraries, bringing the same ease of use to a serverless environment. These also include libraries such as XGBoost, PyTorch, tokenizers, transformers, etc. 

“At Snap we use Spark on Dataproc for a variety of analytics and ML workloads including running GPU accelerated Spark Rapids, and model training and inference with PyTorch. The new Dataproc 2.3 ML runtime has been really helpful — reducing our cluster startup latency by 75% and eliminating toil for our ML Platform developers to build and manage environments.”– Prudhvi Vatala, Sr. Manager, Snap Inc.

It’s easy to create a Dataproc on a compute Engine cluster, specifying the ML image version and the required GPU accelerators for your workers.

Additionally, Serverless Spark sessions (Generally Available) also support GPUs, and come similarly packaged with GPU drivers and common ML libraries.

Develop Spark applications in Colab or your favorite IDE

Now, you can develop and run Spark applications using Colab Enterprise notebooks in BigQuery Studio or via integrated development environments (IDEs) like VSCode and Jupyter. 

BigQuery Colab enterprise notebooks are available within BigQuery Studio with native support for Spark application development. With Colab Enterprise notebooks, you can create  Serverless Spark sessions using Spark Connect and work with your tables in BigLake metastore. 

Colab notebook provides advanced features such as gen AI code assistance and error explanation, with error correction coming soon. It also supports observability for your Spark jobs and management of Spark sessions. In addition, the Colab notebooks lets you mix BigQuery SQL with Spark code in a single notebook and interoperate on the resulting tables. Once your code is ready, you can schedule notebooks via the inbuilt scheduling functionality or use BigQuery Pipelines for more complicated DAGs.

You can also use IDEs such as Visual Studio Code or JupyterLab for Spark application development. JupyterLab users can use the Dataproc JupyterLab plugin to simplify interactive development with Spark serverless sessions and simplify creation and submission of batch jobs via Serverless batch jobs. This plugin comes preinstalled in Vertex Workbench, so you can be productive in minutes.

On VS Code, you can use the Cloud Code extension, which supports development against a range of Google Cloud services. After configuring the Cloud Code extension, you can browse BigQuery datasets and tables, browse and manage your Dataproc compute resources (clusters, serverless interactive sessions and batch), create Spark notebooks from available templates or start developing on your own, and then schedule your workloads all from VS Code. This choice in development tooling allows you to pick one that best suits your workflow, without sacrificing access to the power of Dataproc Spark.

Distributed training and inference with GPU support

Dataproc’s ML runtimes are built to run distributed training and inference, leveraging frameworks like XGBoost, TensorFlow, and PyTorch, all pre-configured for GPU utilization. For example, for distributed training with XGBoost on Spark, you can leverage the pre-installed xgboost.spark library. By setting parameters such as num_workers to distribute the task across Spark executors and device=”cuda”, you can effectively train your models on multiple GPUs, significantly speeding up the training process for large datasets. Here’s an example of how to configure XGBoost classifier for distributed GPU training on your Spark cluster:

Interactive environment customization for Spark Connect

When working interactively with Spark, such as in a Colab notebook using Spark Connect, ensuring Python library consistency between your client and the Spark cluster is crucial. Dataproc simplifies adding PyPI packages dynamically to a Spark session by extending the addArtifacts method. You can now specify the list of packages to install/upgrade/downgrade in version-scheme (same as pip install). This instructs the Spark Connect server to install the package and its dependencies, making them available to workers for your UDFs and other code.

In addition you can also customize your Spark environment on Dataproc with Init scripts and custom images.

MLOps via Vertex AI

Dataproc works with Vertex AI, Google Cloud’s unified platform for AI and ML, helping to improve MLOps for your AI/ML workflows with Spark. Using the Vertex AI SDK directly within your Dataproc Spark code enables experiment tracking and model management, allowing you to:

• Track experiments: Track log parameters, metrics, and artifacts from your Dataproc Spark training jobs to Vertex AI Experiments. This allows you to compare runs, visualize results, and reproduce experiments reliably.

• Register models: Once training is complete, register your trained models into the Vertex AI Model Registry. This provides a central repository for model versioning, staging, and governance, simplifying the path to deployment.

This integration makes your AI/ML workloads on Spark more manageable, reproducible, and deployable, per your organization’s wider MLOps strategy.

Deploy to production

Move from interactive development to production easily.

When using BigQuery Colab notebooks for development, you get Git support to version-control your code and go through your CI/CD flow. You can also schedule your Spark notebook using BigQuery’s built-in pipeline feature, which allows you create single scheduled notebooks or more complicated DAGs, chaining multiple notebooks or queries. You can run these pipelines using the user account or a service account for production pipelines.

BigQuery Pipelines let you compose your flow into discrete tasks, so you can mix Apache Spark on Dataproc and BigQuery execution. In the following BigQuery pipeline, the first task ingests raw data via a BigQuery query, then the data is transformed via Apache Spark via a notebook task. This notebook contains the pertinent Spark transform steps. Finally, the graph splits into two parallel tasks: a notebook that produces a report based on output of the previous task, and a final query that cleans up the initial ingested data.

When using an IDE you can achieve a similar flow by using the Git client of these IDEs to version your Spark code. You can also create and deploy pipelines using Cloud Composer, Google Cloud’s managed serverless Apache Airflow offering. You can run jobs on your existing Dataproc clusters, ephemeral job clusters, or on Serverless Batch.

AI/ML-ready Apache Spark

With Dataproc, you can build your AI/ML workloads with Apache Spark more easily. By providing pre-configured ML Runtimes with GPU support, simplifying Python dependency management for interactive sessions via Spark Connect, enabling development from your preferred IDE, and offering seamless integration with Vertex AI for MLOps, Dataproc accelerates the entire ML lifecycle. Move from exploration and training to robust, production-ready Spark ML pipelines. Explore the Dataproc documentation today to start leveraging these capabilities.

Tzafon, a San Francisco-based startup and AI R&D lab, is partnering with Google Cloud to utilize Google’s AI-optimized infrastructure and cloud services, which will help Tzafon deliver automation at large scale.

The Tzafon team aims to do this by building systems and models that can support multiple, autonomous AI agents that are capable of working together and interacting with common interfaces like applications, operating systems, and web browsers.

Now, Tzafon will partner with Google Cloud to access the compute resources and cloud services it needs to train its new multi-agent models and to develop new automation frameworks that will allow Tzafon’s agents to collaborate more quickly and seamlessly. 

Through its partnership with Google Cloud, Tzafon will:

• Utilize NVIDIA GPUs through Google Cloud to train new machine intelligence models capable of managing multiple AI agents.

• Develop individual agents capable of interacting with operating systems, web browsers, and applications on a person’s behalf.

• Scale workloads up or down quickly using Google Kubernetes Engine.

• Use BigQuery to effectively manage the large volumes of data underpinning its systems.

Today, more than 60% of the world’s generative AI startups are using Google Cloud. Now, Tzafon joins them in gaining access to Google Cloud’s AI stack, with its reliable compute capacity,strong price performance, robust data infrastructure, and elasticity to scale quickly, among many other features that are essential in the emerging field of AI.

You can read more about Tzafon’s mission in their white paper and learn more about how thousands of AI startups are building with Google Cloud here.

It’s funny to think of Renault Group, the massive European car manufacturer, as a software company, but in many ways, it is. Renault Group subsidiary Ampere Software Technology is dedicated to developing and integrating advanced software solutions for intelligent electric vehicles, aiming to create software-defined vehicles (SDVs) with enhanced customer experiences and new services. 

Ampere develops Renault Group’s software-defined vehicle based on large Android Automotive OS (AAOS) codebases. But like all software companies, it struggled to contain costs, sync code bases, maintain adequate testing regimens, and onboard new talent. 

Building on the existing partnership between Google Cloud and Renault Group, Ampere chose a Google Cloud solution for its software-defined vehicle development. This solution, leveraging Google Cloud Workstations and Gemini Code Assist, effectively streamlined the process, making it more secure and productive by eliminating many common development hurdles.

For security-conscious enterprises, Cloud Workstations offer fully managed development environments. Concurrently, Gemini Code Assist, driven by Gemini 2.5, provides secure generative AI coding assistance and agents across the entire software development lifecycle. And by utilizing Google’s virtual twin technology, specifically developed for AAOS, Ampere constructed  full digital counterparts to their automobiles. 

Let’s take a closer look at the components in this solution:

Google Cloud Workstations

Google Cloud Workstations significantly boosts Android Open Source Project (AOSP) developer productivity in general. In the context of Ampere, it offers on-demand development environments with persistent disks, pre-synced with the customer’s AAOS repositories. This eliminates lengthy sync and build times, allowing developers to access their work from anywhere. Ampere’s Platform Admins provision these workstations, drastically cutting down the time it takes for developers to become productive. Developers have instant access to powerful virtual machines with ample vCPUs, RAM, and fast SSD storage — important for the demanding emulators that they use. This resource elasticity prevents bottlenecks and accelerates development. Then, secure, authenticated cloud access and Google Cloud security tools helps to significantly reduce IP leaks and unauthorized access. Finally, having a consistent development environment prevents “works on my machine” problems and reduces debugging time. The  flexible access and disk configurations enhance AAOS developers’ productivity by enabling workstation access from anywhere, and preserving codebases, configurations, and build artifacts across sessions via persistent disks, eliminating repeated repo syncs.

Gemini Code Assist for AI-powered Android development

To help, Ampere offered their developers Android Studio and Code OSS IDEs integrated with Gemini Code Assist, helping to address code management complexity, reduce steep learning curves, and prevent errors. Gemini Code Assist uses retrieval-augmented generation (RAG) to access Ampere’s private codebases and documentation, providing relevant and accurate code suggestions tailored to their Android development standards and conventions. It sped up understanding of their vast codebases by explaining functions, summarizing modules, and suggesting next steps, benefiting new developers and those working on different parts of the SDV software. It also helped boost their Android development productivity by automating boilerplate code, suggesting APIs, and finding potential problems, letting developers concentrate on core SDV logic instead of repetitive tasks.

The virtual twin 

Google Cloud enabled Ampere’s AAOS developers and testers to use a “virtual twin” of a car, resolving resource and complexity issues associated with physical or poorly managed virtual testing. Developers can use powerful Compute Engine instances and specialized Android emulators like Cuttlefish to create accurate virtual vehicle embedded systems. This enables rigorous software testing with virtual hardware, helping to ensure robust performance before building physical prototypes. AAOS developers can also use scalable virtual devices for parallel testing, comprehensive regression suites, and simulations, accelerating the “test” phase of the CI/CD pipeline and improving the SDV lifecycle. The virtual twin is integrated with the Cloud Workstations development environment and the customer’s CI/CD pipelines (e.g., powered by GKE and GitLab), allowing developers to build their AAOS changes on their workstation, trigger automated tests on a fleet of virtual twins, and get immediate feedback on their code.

Tangible returns of modernized SDV development

By combining the robust, managed infrastructure of Cloud Workstations, Gemini Code Assist’s intelligent assistance, and virtual twins, Google Cloud is helping Renault modernize automotive software development, accelerate innovation, and bring new features to market at unprecedented speeds.

“…to invest and build the software platform for the software defined vehicle in Europe ..you need the tools and this is where Google shines …  At the heart of it is AI and when we talk about code generation, instantiation of things that you need to be running immediately versus waiting for the thing to compile and having be available to the developer …  to make our engineers more efficient so we can do more with less time because we are challenged with time.”  – Henry Bzeih, Ampere Chief Software Officer (Renault Group)

Google Cloud and Gemini Code Assist offer automotive OEMs a transformational shift, extending beyond mere tool adoption to significantly impacting business results. This transition enhances competitiveness, profitability, and innovation speed.

Traditionally, onboarding new developers takes days and is costly. AAOS development often involves time-consuming setup, dependency management, and build system troubleshooting. By reducing environment setup time from days to minutes — including repository syncing and toolchain configuration — and utilizing AI assistance, the development process is vastly accelerated.

OEMs worry about intellectual property leaks from local devices. Cloud Workstations addresses this concern by operating within the customer’s Virtual Private Cloud. This approach prevents source code from being synced locally and exposed on endpoints, reducing security risks.

While cloud infrastructure has associated costs, it yields substantial cost optimization. Eliminating high-end local machines, minimizing wasted developer time on environment management, and speeding up timelines all lower total development costs. The ability to quickly adjust cloud resources ensures payment only for active usage, avoiding idle hardware expenses, and a better return on engineering investment.

Finally, having a virtual twin of the car improves quality assurance and validation. Instead of relying on limited prototypes or unreliable local emulators, developers can use detailed virtual car models, facilitating faster iteration, scalable testing, early bug detection, and advanced scenario simulation.

Automotive companies are not only adopting new technologies but are also reshaping their development capabilities by utilizing Cloud Workstations and Gemini Code Assist. For more, watch the fireside chat with Henry Bzeih, Ampere Chief Software Officer (Renault Group) on their success with this SDV Industry solution.

And if you’re in the automotive industry, you can get started on setting up custom AAOS development environments with Gemini Code Assist by referring to our GitHub repository. And of course,  your Google Partner Engineering or Customer Engineering contacts are ready to help!

Search is a critical component of many modern applications – whether searching for products in an online storefront, finding solutions to your customers’ support cases, or building the perfect playlist. But traditional keyword searches often miss the deeper meaning of data. Vector embeddings, however, capture the complexities of your data, enabling highly accurate and powerful searches. Vector embeddings are numerical representations of your data, generated by models, that help computers understand nuances and meaningfully compare data points.

Cloud SQL for MySQL’s generally available vector support allows you to store vector embeddings, build persistent indexes on them, and perform ANN search between them. Now, we have a new Vertex AI integration that makes this process easy by helping you generate vector embeddings using a simple SQL function, eliminating the need for external embedding generation. You can also leverage any Vertex AI model directly from MySQL, bringing advanced AI capabilities closer to your data.

Ready to see how AI can improve searches in your application and help you understand and predict customer trends? Let’s dive in.

Generate, store, & search vector embeddings

Let’s say you have an ecommerce application and are looking to see how Vertex AI and vector search with Cloud SQL for MySQL can power product searches and help you analyze customer reviews.

After setting up Vertex AI permissions and enabling the feature, you can start embedding your existing product data with the `mysql.ml_embedding` function. You just need to specify the model name and then pass the text you would like to embed–all formatting and parsing of Vertex AI requests is handled for you. 

For example, in your ecommerce application you can use a query like the following to directly store the output of `mysql.ml_embedding` in a VECTOR type column:

To quickly find similar products, create a vector index for ANN search. You can select the best distance measure for your use case and customize the number of leaves to fine-tune speed and recall.

When a customer searches in your application, you can embed their query using `mysql.ml_embedding` and pass it to ANN search. For example, to find clothes for a specific occasion in your database:

Get responses from Gemini & other Vertex AI models

The Cloud SQL Vertex AI integration lets you make requests to any Vertex AI endpoint, including custom models fine-tuned to your datasets or pre-trained models like Gemini. This means that in addition to being able to generate vector embeddings for product descriptions, you can use Vertex AI to make predictions about things like what your users will purchase or whether a purchase is fraud.

For example, say you store customer reviews of your products in a MySQL database and want to perform analysis on them to get an idea of customer sentiment. You can invoke a Vertex AI LLM to determine customer satisfaction directly from the MySQL client and store its assessment.

Start building

Want to get started with the MySQL Vertex AI integration after seeing how it enhances product search and analysis? Explore our detailed codelab for an embeddings workflow tutorial or dive into our documentation for a comprehensive overview of all capabilities.

Editor’s note: This is the first in a series of five blog posts dedicated to data transformation powered by Google Cloud and its ecosystem of data and analytics partners. 

Everyone is trying to determine how best to leverage new AI technologies. But to be able to get the most out of AI, you need a strong data foundation. For that, you need to get your house in order.

The challenges around data aren’t new. We’ve always known that data has immense potential, but actualizing it has been persistently difficult. Initially, the conversations were about accumulation — building bigger and better data repositories. Then the focus shifted to extracting insights from that data, with a proliferation of analytics tools and techniques. Yet we still grapple with multiple challenges like data silos, quality concerns, and complexity. In fact, even with sophisticated tools, vast amounts of data remain untapped and underutilized. According to a 2024 survey by Wavestone, while 82% of organizations are increasing their investments in data and analytics, fewer than half consider themselves industry leaders in this domain. Clearly, it’s time for a new chapter in the data narrative. 

True data leadership

True data leadership demands a holistic approach to data, one that prepares us for an AI-powered future. It requires enterprise intelligence. Enterprise intelligence is not just about having data. It’s about activating it strategically to drive smarter decisions, enhance customer experiences, guide product innovation, and ultimately fuel business growth.

What is enterprise intelligence?

Enterprise intelligence refers to the ability of an organization to effectively use data, analytics, and advanced machine learning (ML) and AI technologies to make informed decisions, optimize operations, and drive innovation. It represents a holistic approach to data management that integrates people, processes, and technology by mastering four data capabilities in sequence:

1. Open data access

2. Unified insights

3. Mature AI/ML analytics

4. Strong data culture

Let’s take a deeper dive into each of these capabilities. With open data access, you’re breaking down silos and enabling the seamless flow of information across your organization. Google Cloud partners like Fivetran and Confluent support this with solutions for data integration and real-time data streaming. 

Once you’ve achieved open data access, unified insights emerge, creating a single source of truth. Partners such as Elastic, MongoDB, and Neo4j play a crucial role in this step, offering solutions for flexible data storage and centralized data management to help uncover complex relationships within your data. With this solid foundation in place, you are now positioned to leverage mature AI/ML analytics to extract deeper insights, predict future trends, and automate complex processes, driving innovation and efficiency. Partners like Databricks excel in this area, providing powerful platforms for unified analytics, AI model development, and real-time data processing. 

Finally, a strong data culture ensures that data is not just an asset but a core part of the organization’s DNA, with data embedded into the very fabric of the way a company operates. Collibra strengthens your data culture by enabling both business and technical users to discover, understand, and trust data across hybrid and multi-cloud environments — fueling collaboration and scalable decision-making. 

Insights for AI

Enterprise intelligence is the necessary foundation to fuel AI-driven growth. However, achieving this level of data maturity requires a strategic blend of the right tools, technologies, and expertise. Google Cloud and its robust data partner ecosystem provide a comprehensive suite of solutions and services, along with specialized integrations. 

The choice is yours. In today’s rapidly evolving landscape, those who fail to master enterprise intelligence risk being left behind. Over the next four blogs, we’ll delve more deeply into the four steps you need to take your organization to true data-driven enlightenment — and ready to harness the full potential of your data and AI. 

Ready to get started?

Take this short assessment to see where you stand in enterprise intelligence. The future is data-driven and AI-powered — let’s build it together.

The excitement around open Large Language Models like Gemma, Llama, Mistral, and Qwen is evident, but developers quickly hit a wall. How do you deploy them effectively at scale? 

Traditional load balancing algorithms fall short, as they fail to account for GPU/TPU load status, leading to inefficient routing for computationally intensive AI inference with its highly variable processing times. This directly impacts serving performance and the user experience.

This guide demonstrates how Google Kubernetes Engine and the new GKE Inference Gateway together offer a robust and optimized solution for high-performance LLM serving, specifically by overcoming the limitations of traditional load balancing with smart routing aware of AI-specific metrics like pending prompt requests and critical KV Cache utilization.

We’ll walk through deploying an LLM using the popular vLLM framework as the inference backend. We’ll use Google’s gemma-3-1b-it model and NVIDIA L4 GPUs as a concrete, easy-to-start example (avoiding the need for special GPU quota requests initially). The principles and configurations shown here apply directly to larger, more powerful models and diverse hardware setups.

Why Use GKE Inference Gateway for LLM Serving?

GKE Inference Gateway isn’t just another ingress controller; it’s purpose-built for the unique demands of generative AI workloads on GKE. It extends the standard Kubernetes Gateway API with critical features:

• Intelligent load balancing: Goes beyond simple round-robin. Inference Gateway understands backend capacity, including GPU-specific metrics like KV-Cache utilization, to route requests optimally. For LLMs, the KV-Cache stores the intermediate attention calculations (keys and values) for previously processed tokens. This cache is the primary consumer of GPU memory during generation and is the most common bottleneck. By routing requests based on real-time cache availability, the gateway avoids sending new work to a replica that is near its memory capacity, thus preventing performance degradation and maximizing GPU usage, increasing throughput, and reducing latency.
• AI-aware resource management: Inference Gateway recognizes AI model serving patterns. This enables advanced use cases like serving multiple different models or fine-tuned variants behind a single endpoint. It is particularly effective at managing and multiplexing numerous LoRA adapters on a shared pool of base models. This architecture dramatically increases model density on shared accelerators, reducing costs and operational complexity when serving many customized models. It also enables sophisticated, model-aware autoscaling strategies (beyond basic CPU/memory).
• Simplified operations: Provides a dedicated control plane optimized for inference. It seamlessly integrates with GKE, offers specific inference dashboards in Cloud Monitoring, and supports optional security layers like Google Cloud Armor and Model Armor, reducing operational overhead.
• Broad model compatibility: The techniques shown work with a wide array of Hugging Face compatible models.
• Flexible hardware choices: GKE offers access to various NVIDIA GPU types (L4, A100, H100, etc.), allowing you to match hardware resources to your specific model size and performance needs. (See GPU platforms documentation).

The Walkthrough: Setting Up Your Inference Pipeline

Let’s get started building out our inference pipeline. By following these steps, you will deploy and configure the essential infrastructure to serve your LLMs with the high performance and scalability demanded by real-world applications, built on GKE and optimized by the Inference Gateway.

Environment Setup

Ensure your Google Cloud environment is ready. All steps in this walkthrough are tested in Google Cloud Shell. Cloud Shell has the Google Cloud CLI, kubectl, and Helm pre-installed.

1. Google Cloud project: Have a project with billing enabled.

2. Google Cloud CLI: Ensure gcloud is installed and updated. Run gcloud init if needed.

3. kubectl: Install the Kubernetes CLI: gcloud components install kubectl

4. Helm: Install the Helm package manager (Helm installation guide).

5. Enable APIs: Activate necessary Google Cloud services.

6. Configure permissions (IAM): Grant required roles. Remember to follow the principle of least privilege in production environments.

7. Set region: Choose a region with the GPUs you need.

8. Hugging Face token: Obtain a Hugging Face access token (read permission minimum). If using Gemma models, accept the license terms on the Hugging Face model page.

Create GKE Cluster Resources

Set up the GKE cluster and necessary networking components.

1. Proxy-only subnet (run once per region/VPC): Required for Inference Gateway’s regional load balancer.

2. GKE standard cluster: Inference Gateway currently requires a Standard cluster.

3. Configure kubectl:

4. Accelerator node pool: Add nodes with GPUs. Ensure you have quota for the chosen GPU type and the zone supports it.

Install Gateway API and Inference Gateway CRDs

Apply the Custom Resource Definitions (CRDs) that define the necessary Kubernetes objects.

NOTE: Using kubectl apply with remote URLs means you’re fetching the manifests at execution time. For production, consider vendoring these manifests or referencing specific tagged releases.

NOTE: You might see warnings about missing annotations if GKE pre-installed some base CRDs; these are generally safe to ignore during initial setup.

Deploy the LLM Inference Server (using vLLM)

First, create the Kubernetes Secret to securely store your Hugging Face token, which the deployment will need to download the model.

Now, define and apply the Kubernetes Deployment for the pods running the vLLM server with our chosen model. Inference Gateway will route traffic to these pods.

Key configurations in the YAML:

• metadata.labels.app: Crucial! The InferencePool will use this label to find the pods.

• spec.template.spec.containers[0].resources: Must match the GPU node pool (e.g., nvidia.com/gpu: "1" for one L4).

• spec.template.spec.containers[0].env.MODEL_ID: Set to the Hugging Face model ID.

• spec.template.spec.nodeSelector: Ensures pods land on the GPU nodes.

• spec.template.spec.containers[0].*Probe: Vital for health checks and readiness signals to the Gateway.

Save as llm-deployment.yaml:

Apply the Deployment and wait for the pod(s) to become ready. This includes the time needed to download the model, which can take several minutes depending on model size and network speed.

Configure GKE Inference Gateway Resources

Now, define how the Inference Gateway manages traffic to the deployed model server pods.

1. Create the inference pool: This resource groups the backend pods using the labels defined in the Deployment. We use the official Helm chart for this.

2. Define the inference model: Specifies metadata about the model served by the pool. Save as gemma-3-1b-inference-model.yaml:

3. Apply: kubectl apply -f gemma-3-1b-inference-model.yaml

4. Define the entry point: The gateway: Creates the actual load balancer. Save as inference-gateway.yaml:

5. Apply: kubectl apply -f inference-gateway.yaml Load balancer provisioning takes a few minutes.

6. Route the traffic: The HTTPRoute: Connects requests coming into the Gateway to the correct InferencePool based on path matching. Save as gemma-3-1b-httproute.yaml:

7. Apply: kubectl apply -f gemma-3-1b-httproute.yaml

Verify the Deployment

Let’s check if everything is wired up correctly.

1. Get gateway IP address: Wait for the load balancer to get an external IP.

2. Send test inference request: Use curl to send a request to the Gateway endpoint.

NOTE: This uses HTTP/80 for simplicity. Production requires HTTPS/443.

If successful, you’ll receive an HTTP/1.1 200 OK status followed by a JSON response containing the LLM’s output. If you encounter issues, check the Gateway status (kubectl get gateway ... -o yaml) and the logs of your vLLM pods (kubectl logs deployment/gemma-3-1b-deployment).

Take Your LLM Serving to the Next Level

You’ve successfully deployed an LLM behind the GKE Inference Gateway! Now it’s time to explore its powerful features to build truly production-ready systems:

• Scale smartly with autoscaling: Don’t guess capacity! Configure a HorizontalPodAutoscaler (HPA) for your gemma-3-1b-deployment. Scale based on the inference_pool_average_kv_cache_utilization metric provided by Inference Gateway. This ensures you scale based on actual AI workload demand, not just CPU/memory.
• Gain visibility with monitoring: Keep a close eye on performance. Use the dedicated Inference Gateway dashboards in Cloud Monitoring to track request counts, latency, error rates, and KV Cache metrics at the gateway level. Combine this with backend pod metrics (GPU utilization, vLLM stats) for a complete picture.
• Expand your model portfolio: Serve multiple models efficiently. Deploy other models (e.g., Llama 4, Mistral, or your own fine-tuned variants) using separate Deployments and InferencePools. Use advanced HTTPRoute rules (path-based, header-based, or even request-body-based routing via ExtensionRef) to direct traffic to the correct model pool, all behind the same Gateway IP.
• Bolster security and reliability: Protect your endpoints. Configure HTTPS on your Gateway listener using Google-managed or custom TLS certificates. Apply Google Cloud Armor policies at the load balancer for robust WAF and DDoS protection. Consider integrating Model Armor for content safety filtering via GCPTrafficExtension.
• Deploy larger, more powerful models: Ready for the big leagues? For models like Qwen 3 235B,, select appropriate GPUs (A100, H100), significantly increase resource requests/limits in your Deployment, adjust vLLM parameters (like tensor-parallel-size), and potentially increase probe timeouts. Inference Gateway’s intelligent load balancing and management become even more critical for efficiently handling these demanding workloads.

By combining the capabilities of modern LLMs with the operational power of GKE and Inference Gateway, you can build, manage, and scale sophisticated AI applications effectively on Google Cloud. Dive deeper into the official GKE Inference Gateway documentation for comprehensive configuration details and advanced scenarios.

Embeddings are a cornerstone of modern semantic search and Retrieval Augmented Generation (RAG) applications. In short, they enable applications to understand and interact with information on a deeper, conceptual level. In this post, we’ll show you how to create and retrieve embeddings with a few lines of Dataflow ML code to enable both of these use cases. We will cover streaming and batch approaches for generating embeddings and storing them in vector databases such as AlloyDB  to power semantic search and RAG applications with their vector search capabilities.

Semantic search and RAG

In semantic search, we are able to leverage underlying relationships between words to find relevant results beyond a simple keyword match. Embeddings are vector representations of data, ranging from text to videos, that capture these relationships. In a semantic search we find embeddings that are mathematically close to our search query, allowing us to find words or search terms close in meaning that may have not shown up in a keyword search. Databases such as AlloyDB allow us to combine this unstructured search with a structured search to provide high quality relevant results. For example, the prompt “Show me all pictures of sunsets I took in the past month” includes a structured part (the date is within the past month) and an unstructured part (the picture contains a sunset).

In many RAG applications, embeddings play an important role in retrieving the relevant context from a knowledge base (such as a database) to ground the responses of large language models (LLMs).  RAG systems can perform a semantic search on a Vector Database, such as AlloyDB, or directly pull data from the database to provide the retrieved results as context to the LLM so that it has access to the necessary information to generate informative answers. 

Knowledge ingestion pipelines

A knowledge ingestion pipeline takes unstructured content, such as product catalogs with free form descriptions, support ticket dialogs, and legal documents, processes them into embeddings, and then pushes these embeddings into a vector database. The source of this knowledge can vary widely, from files stored in cloud storage buckets (like Google Cloud Storage) and information stored in databases like AlloyDB, to streaming sources such as Google Cloud Pub/Sub or Google Cloud Managed Service for Kafka. For streaming sources, the data itself might be raw content (e.g, plain text) or URIs pointing to documents.  A key consideration when designing knowledge ingestion pipelines is how to ingest and process knowledge, whether in a batch or streaming fashion. 

• Streaming vs Batch: To provide the most up-to-date and relevant search results, and thus a superior user experience, embeddings should be generated in real time for streaming data. This applies to new documents being uploaded or new product images, where current knowledge holds significant business value. For less time-sensitive applications and operational tasks like backfilling, a batch pipeline is suitable. Crucially, the chosen framework must support both streaming and batch processing without requiring business logic re-implementation.

• Chunking: Regardless of the data source, after reading the data there is normally a step for preprocessing the information. For simple raw text, this might mean basic cleaning. However, for larger documents or more complex content, chunking is a crucial step. Chunking breaks down the source material into smaller, manageable units. The best chunking strategy varies depending on the specific data and application. 

Introducing Dataflows MLTransform for embeddings

Dataflow ML provides many out of the box capabilities to simplify the entire process of building and running a streaming or batch knowledge ingestion pipeline, allowing you to implement these pipelines in a few lines of code. For an ingestion pipeline there are typically four phases, reading from data sources, preprocessing the data, making it ready for embeddings, and finally writing the correctly shaped schema to our vector database. The new capabilities in MLTransform adds support for chunking, generation of embeddings, using Vertex or bring your own (BYO) models and specialized writers for persisting embeddings to databases such as AlloyDB.

With Dataflow’s new MLTransform capabilities, the flow to chunk and generate embeddings and land them into AlloyDB can be achieved within a few lines of code:

Importantly, for the embeddings step shown above in Dataflow ML supports the ability to make use of Vertex AI embeddings as well as models from other model gardens and the ability to “bring your own model (BYOM)” hosted on Dataflows workers. 

The example above creates a single chunk per element, but you can also use LangChain for chunking instead:

You can find a full working example with AlloyDB here: build realtime vector embedding pipeline for AlloyDB with Dataflow. 

Introducing Dataflow enrichment transform for RAG

To support RAG use cases, we have also enhanced Dataflows enrichment transform to include the ability to look up results from databases, enabling the ability to create asynchronous applications in batch or stream mode that can, in a low code pipeline, consume the information stored in the vector database. This enables you to enrich your applications with operational data to embed or to utilize as filters, so that you don’t need to store your vectors in distinct storage solutions.

Get started today

With these simple code snippets, we’ve shown how it’s possible to not only ingest and prepare the source data needed to populate your vector databases, but also to consume and use that information for your streaming and batch applications where the intent is to process large volumes of data and incoming information. For a full working example please look at the following full example. To explore more please head to Dataflow ML documentation for detailed examples and collabs. Get started with vector search on AlloyDB today. You can also sign up for a 30-day AlloyDB free trial.

^{Special thanks to Claude van der Merwe, Dataflow ML Engineering, for his contributions to this blog post.}

Imagine transforming a sprawling, 20-year-old telecommunications data ecosystem, laden with sensitive customer information and bound by stringent European regulations, into a nimble, cloud-native powerhouse. That’s precisely the challenge Deutsche Telekom tackled head-on, explains Ashutosh Mishra. By using Google Cloud’s Sovereign Cloud offerings, they’ve built a groundbreaking “One Data Ecosystem.”

When we decided to modernize our telecommunications data ecosystem at Deutsche Telekom, we faced a daunting task. Over 40 legacy systems, each an ecosystem (data warehouse or data lake), held terabytes of customer, network, and operational data. Each system had 5,000 to 10,000 users who had built their workflows around these isolated silos over decades of use. 

The result? What I lovingly call a “spaghetti mess” of data distribution and an undetermined cost of value creation.

The technical challenge of building our One Data Ecosystem (ODE) was one thing — consolidating disparate systems always is. It’s the regulatory puzzle that made it genuinely interesting. 

As a telecommunications company in Germany, we handle some of the most sensitive data imaginable: call data records (CDRs), network telemetry, and customer location data. Under GDPR and Germany’s Telecommunications and Telemedia Data Protection Act regulations, this data simply can not leave sovereign borders or risk exposure to foreign legal frameworks.

Here’s where it gets technically fascinating: Traditionally, regulated industries solve this by building expensive on-premises encryption and pseudonymization infrastructure. You process your sensitive data locally, strip it of identifying characteristics, and then send sanitized versions to the cloud for analytics. 

This approach costs millions in dedicated hardware and creates a fundamental innovation bottleneck. We wanted something radically different: cloud-native processing of sensitive data, without compromise.

Engineering sovereignty at cloud scale

The breakthrough came with Google Cloud’s approach to digital sovereignty and their Germany Data Boundary by T-Systems offering (formerly known as Sovereign Controls by T-Systems). The architecture is elegant in its simplicity: Deutsche Telekom maintains complete cryptographic control through external key management (EKM) while using cloud-native data services.

Here’s how the technical magic works. T-Systems manages our encryption keys entirely outside Google’s infrastructure. This creates sovereign protection against foreign legal frameworks, and ensures they are able to control access to their data and deny access for any reason. 

Meanwhile, we use format-preserving encryption (FPE) algorithms that maintain data utility for analytics while ensuring privacy protection.

The core innovation is our custom pseudonymization layer, which comprises C++ modules with Java wrappers that handle real-time data transformation during ingestion. This eliminates the traditional need for separate preprocessing infrastructure while maintaining analytical value.

Choosing our data format was crucial. After extensive POCs, we settled on Apache Iceberg, and here’s why that matters for anyone building similar platforms: Iceberg solves the polyglot analytics problem beautifully. Our data scientists prefer working in Python notebooks, our engineers use Spark, and our business analysts work with SQL.

While traditional approaches force you to pick sides or maintain multiple data copies, Iceberg provides us with a single source of truth that speaks every language fluently.

The three-layer architecture we built around Iceberg is worth replicating: Raw data lands directly in Cloud Storage, flows through an Atomic layer for normalization and schema evolution, and then surfaces in an Analytic layer optimized for specific use cases. BigQuery, Spanner, BigTable, and Cloud SQL each serve their optimal workloads while sharing the same underlying Iceberg foundation.

Performance and scale in production

We are migrating from more than 40 legacy systems to reinvigorate our business demands. We have ingested over 200 source systems in just six months. However, the real validation came recently when one of our use cases, running live on the new platform, achieved a 22x performance improvement over its legacy predecessor. 

That number represents the compound effect of eliminating data silos, reducing ETL complexity, and using cloud-native autoscaling. When you can process overnight analytics jobs in minutes instead of hours, you fundamentally change how business decisions get made.

What makes this platform genuinely scalable isn’t just the technical architecture; it’s the operational model. We’ve implemented a GitOps approach with policy-as-code onboarding through GitLab CI/CD pipelines, where infrastructure and governance policies are defined declaratively and deployed automatically. This means onboarding a new system takes hours instead of months, and compliance becomes automatic rather than manual.

Additionally, we’re already running agentic AI use cases on the public side of our platform. The unified data model we’ve built positions us perfectly for the next wave of AI innovation. As more AI services become available with sovereign controls, we’ll be ready to expand our deployment at scale.

The key insight: Build your data foundation with AI in mind, even if you can’t implement every AI capability immediately. Clean, unified, well-governed data is the prerequisite for everything that’s coming.

A blueprint for the future

This is one of the largest and most comprehensive data platforms built on Google Cloud’s Data Boundary — but it won’t be the last. The architectural patterns we’ve developed, external key management, format-preserving encryption, unified data formats, policy-as-code governance, are replicable across any regulated industry.

The business case is also compelling: Eliminate expensive on-premises preprocessing infrastructure while gaining cloud-scale analytics capabilities. The technical implementation is proven. What’s needed now is the willingness to engineer sovereignty, rather than simply accept traditional trade-offs.

For my fellow data architects in regulated industries, you don’t have to choose between innovation and compliance. With the right technical approach, you can achieve both and build platforms that position your organization for the AI-driven future that’s rapidly approaching.

The maturity and integration of Google Cloud’s data and AI capabilities, combined with our intensive collaboration between engineering teams, has made this transformation possible. We’re not just customers: We’re co-creating the future of sovereign cloud platforms.

In the world of database management, understanding performance bottlenecks is critical to smooth operations and an optimal user experience. Using managed database services can help alleviate mundane management tasks and let you focus on value-added, strategic tasks, while also offering tools for monitoring database and resource performance. But when performance issues arise, gathering and analyzing various system and database metrics before and during the incident to conduct a thorough analysis can be time-consuming and labor-intensive. 

Google Cloud’s AlloyDB for PostgreSQL is a fully managed database service that delivers superior availability, scalability, and performance. Recently, we added a new feature called performance snapshot report that provides deep insights into database performance. In this blog post, we explore how the performance snapshot report lets you identify and solve performance issues with greater clarity, control, and ease. This tool complements other AlloyDB observability features like systems insights, query insights, and the Metrics Explorer, which provide real-time metrics about your instance.

Demystifying the performance snapshot report

The performance snapshot report is a collection of PostgreSQL functions and views that captures snapshots of your AlloyDB system statistics. These snapshots provide detailed diff reports of performance metrics between different points in time, offering a comprehensive view of your database’s activity during the period.

Here’s how it works

The performance snapshot report captures snapshots of your database’s performance metrics at any point during workload execution and generates reports based on any two snapshots. This gives you granular control over what periods you want to analyze, whether it’s the entire workload, specific intervals, or moments of peak activity. Each snapshot captures a wealth of information, including changes in key performance indicators like wait events, I/O statistics, and query execution times. It also captures system configuration details such as CPU usage, memory allocation, and parameter details.

You can also access and analyze performance by generating reports between two snapshots directly within your PostgreSQL environment, using familiar psql queries, as in familiar Oracle AWR reports, facilitating easy adoption and integration into existing processes. Then, you can save the reports for historical comparisons to identify regressions, evaluate changes, and monitor performance trends. 

Performance snapshot report in action

The performance snapshot report equips you with the tools you need to diagnose and resolve performance issues effectively. Let’s explore some common scenarios where a performance snapshot report proves to be invaluable.

1. Identifying performance bottlenecks

Scenario: The Cymbal Shop’s database, managed by DBA Sarah, is experiencing an unexpected performance degradation. A previously stable system became significantly slower this morning. Sarah needs to investigate this sudden slowness to determine its cause and implement a solution to address the changes that led to it.

Solution: Sarah had already established a baseline by creating a report comparing two performance snapshots taken two days prior during normal system operation. To investigate the unexpected slowdown, Sarah first uses the `perfsnap.snap()` function to capture performance snapshots at two distinct time points, separated by approximately one hour. Subsequently, she utilizes the `perfsnap.report(start_snap_id, end_snap_id)` function to generate a differential report, which highlights variations in critical performance indicators such as wait events, I/O activity, and query execution times. Finally, Sarah analyzes the generated report, focusing on significant increases in particular wait events — such as `lwlock` for lock contention or `io` for storage delays — and changes in query execution durations. This detailed examination allows her to precisely identify the root cause of the sudden performance degradation.

Example:

That generates a report similar to this example report.

2. Assessing performance impact post system upgrades and configuration changes

Scenario: Sarah has upgraded the AlloyDB version. She’s also modified a few configuration parameters. She wants to understand the system behavior post these changes.

Solution:

1. Make baseline snapshot report: Before making the changes, Sarah captured two snapshots to create a report that shows the baseline stats.

2. Create post-change snapshot report: After making the changes, she captured two snapshots, and generated another report. 

3. Analyze changes: She compared the report for any significant changes in performance metrics by comparing the two reports. Pay attention to sections like “CPU Utilization,” “Memory Utilization,” and “I/O Statistics” to assess the impact of the changes.

Know thy database performance

The performance snapshot report is a convenient way for you to get visibility and control over your AlloyDB database performance. By capturing and comparing snapshot reports, you can identify bottlenecks, optimize workloads, assess changes, and diagnose performance regressions with ease. This powerful tool, combined with the inherent performance advantages of AlloyDB, empowers you to unlock the full potential of your database and ensure your applications run smoothly and efficiently. 

If you’re new to Google Cloud and want to take AlloyDB for a spin, just sign up for an AlloyDB free trial. If you’re already a Google Cloud user, head over to the AlloyDB console, click “Create a trial cluster” and we’ll guide you through migrating your PostgreSQL data to a new AlloyDB database instance.

Learn more 

• Review the documentation for the performance snapshots rReport

• Watch this session for a deep dive into the latest innovations in AlloyDB

• Learn more about AlloyDB

Editor’s note: The Jina AI Reader is a specialized tool that transforms raw web content from URLs or local files into a clean, structured, and LLM-friendly format.  In this post, Han Xiao details how Cloud Run empowers Jina AI to build a secure, reliable, and massively scalable web scraping system that remains economically viable. This post explores the collaborative innovation, technical hurdles, and breakthrough achievements behind Jina Reader, a web grounding system now processing 100 billion tokens daily.

When Jina Reader launched in April 2024, its explosive growth — serving more than 10 million requests and 100 billion tokens daily — confirmed huge demand for reliable, LLM-friendly web content. Jina Reader isn’t just another scraper; it takes a different approach to  how AI systems consume web content by transforming raw, noisy web pages into clean, structured markdown.

The core challenge for any AI system processing web data is the “web grounding problem.” Modern websites are a chaotic mix of content, ads, tracking scripts, and dynamic JavaScript, creating an overwhelming noise-to-signal ratio. Traditional scrapers struggle with this complexity, often failing on dynamic single-page applications or generating unusable, ungrounded data for LLMs. Jina Reader’s breakthrough, ReaderLM-v2, is a purpose-built 1.5-billion-parameter language model that intelligently extracts content, trained on millions of documents to understand web structure beyond simple rules.

Cloud Run: The engine behind Jina Reader’s scale

Jina Reader faced  inherent burstiness and unpredictability of web scraping workloads. Traditional virtual machine setups meant either costly over-provisioning or critical failures under load. Google Cloud Run became the essential solution, enabling Jina Reader to build a web scraping system that is secure, reliable, massively scalable, and economically viable.

• The web grounding app (the browser automation system that scrapes and cleans web content) is hosted on Cloud Run (CPU). It runs full Chrome browser instances.

• ReaderLM-v2 is a purpose-built 1.5-billion-parameter language model for HTML-to-markdown conversion that runs on Cloud Run with serverless GPUs.

Cloud Run directly addressed several critical issues:

• Optimized Performance: The deep collaboration between Jina Reader and Google Cloud engineering was essential. We jointly optimized container lifecycle management for browser automation, reducing startup times from over 10 seconds to under two seconds  through prewarming, optimized images, and intelligent resource allocation. For ReaderLM-v2, Google’s team helped create custom container configurations to efficiently run a 1.5-billion-parameter model on Cloud Run GPUs. The on-demand scaling and fast start capabilities of Cloud Run GPUs were critical in helping optimize model performance, directly impacting our ability to process 100 billion tokens daily.

• True Scale-to-Zero Serverless: Cloud Run’s ability to run full Chrome browser instances allowed cost-effective operations. Each request spawns an isolated container with its own headless Chrome, and crucially, these containers disappear when the request is done. This ephemeral nature is vital for processing untrusted web content, mitigating security risks and memory leaks.

• Global Multi-Regional Deployment: Cloud Run’s global presence ensures requests are processed close to both the users and target websites. This significantly minimizes latency and boosts success rates, even against geo-restricted content.

• Massive & Automatic Scaling: The platform seamlessly scales from a handful to over 1,000 container instances during peak traffic, handling the unpredictable nature of web scraping without manual intervention.

• Economic Viability: With Cloud Run’s pay-per-use model, Jina Reader can offer a generous free tier to end users while maintaining profitability even with substantial monthly usage. This pricing flexibility was fundamental to our widespread adoption.

• Resilience and Operational Excellence: During a recent sustained DDoS attack, Cloud Run’s serverless architecture proved invaluable. It scaled up to absorb massive loads (over 100,000 requests per minute), while intelligent rate limiting filtered malicious traffic. Critically, costs returned to normal immediately after the attack subsided due to its scale-to-zero capability.  The system has maintained over 99.9% uptime.

Conclusion

Building Jina Reader on Google Cloud Run proved that AI capabilities and cloud-native architecture are complementary. Cloud Run’s unique capabilities — serverless GPUs, container isolation, global deployment and scale-to-zero economics — made the architecture possible. Our close partnership demonstrates that deep integration between AI-first systems and modern cloud infrastructure can create capabilities previously thought impossible, enabling us to process 100 billion tokens every day.

You can discover more about Cloud Run GPUs on our product page, and if you want to learn how to host a large language model on Cloud Run, watch this video.

As one of India’s largest healthcare providers, Manipal Hospitals serves nearly 7 million patients annually across 37 hospitals. To deliver clinical excellence and patient-centric care at a high standard, we are continually embracing technology.  One of our most significant operational challenges we consistently face is the nurse handover process—a critical but time-consuming task. To make nurse handovers more efficient, safe and accurate, we entered a strategic partnership with the Google Cloud Consulting team to co-develop a generative AI solution, leveraging the power of  Google Cloud.

Rethinking time-consuming, error-prone nurse handoffs

The process of transferring essential information about a patient’s condition and care plan from an outgoing nurse to an incoming one is crucial for ensuring continuity of care and patient safety. However, with more than 10,500 beds across our hospitals, the sheer volume of data required for a comprehensive handover meant our nurses routinely added an extra 90 minutes to their shifts for both creating and receiving these reports. This lengthy process could directly affect patient care, as it could lead  to fatigue and potential mistakes, and also reduce job satisfaction for our vital nursing staff. We needed a way to make this process faster, more accurate, and less of a burden.

Building a trusted solution on Google Cloud

Our joint Manipal-Google team knew that for a clinical tool to be adopted by over 5,000 nurses, it had to be both fast and trustworthy. The primary challenge with any generative AI application in healthcare is ensuring accuracy and minimizing the risk of AI “hallucinations.”

The solution’s architecture, designed by the Google Cloud Consulting team, addresses this head-on by leveraging multiple Google Cloud components. Patient data from our TrakCare system is securely transferred in near real-time to a data lake on Google Cloud. When a nurse requests a handover summary, a serverless Cloud Run application orchestrates a multi-stage process.

Critically, instead of passing pages of raw data directly to the AI, the system first uses intelligent, time-based filters to extract only the most relevant clinical information for the specific shift. This structured, pre-processed data is then sent to Gemini on Vertex AI. This “controlled generation” approach was a key innovation; it ensures Gemini summarizes only the most pertinent facts, dramatically improving the accuracy and consistency of the final ISBAR (Identify, Situation, Background, Assessment, and Recommendation) report. The ability of Gemini to understand complex medical terminology, medication names, and clinical procedures without specialized fine-tuning was a game-changer, accelerating the entire development process.

How our partnership delivered results

By combining Manipal’s deep clinical expertise and Google Cloud Consulting’s technical leadership, our joint approach provides a blueprint for enterprise-grade AI implementation:

• From ideation to production: The Google Cloud Consulting team led the engagement from the initial idea all the way to a production-ready solution now used by thousands of nurses daily. The project started with a focused Minimum Viable Product (MVP) to prove the technology’s value before scaling.

• User-centric design: The solution was not built in a vacuum. The Google team conducted over eight rounds of deep discussion and evaluation sessions directly with our nurses. This ensured the final ISBAR summary format was not just technically impressive, but clinically useful from day one.

• Agile and iterative rollout: The solution was piloted at one hospital initially to test its performance and safety in a real-world setting. With a successful pilot, the solution is live in 23 of Manipal hospitals, and used by more than 5,000 every day. At full scale, it is projected to help save significant nurse hours on a daily basis.  This phased approach, managed jointly, has allowed us to gather feedback and ensure smooth adoption.

Ensuring better patient care

The generative AI solution we implemented has yielded impressive returns. The 70% reduction in handoff time—from 90 minutes down to 20—frees our nurses to focus more on direct patient needs and care. It also makes the process less vulnerable to errors that can arise from handwritten notes and human fatigue.

This project, delivered in partnership with Google Cloud Consulting, is a prime example of how we are pioneering the future of healthcare in India, helping us scale the delivery of quality care across the length and breadth of the country.

^{We’d like to give special thanks to Google Cloud Consulting team –  Naveen Poosarla, Gopala Dhar, Rupjit Chakraborty, Hem Anand, Amit Dutta, Nishant Welpulwar, Preetam Dey and Shikha Saxena – for designing and developing the solution. We are grateful to the Manipal Hospitals team – Saroja Jaykumar, Sunil Bhattacharjee –  in delivering this successful project.}

As Google Kubernetes Engine (GKE) deployments grow and scale, adopting a multi-project strategy in Google Cloud becomes a best practice for security and environment organization. Creating clear boundaries by using distinct projects for development, testing, and production environments provides isolation and helps manage access control.

However, isolation introduces a data protection challenge: How do you effectively manage backups across these project boundaries? Without a native solution, centralizing backups, ensuring a clear separation of duties with IAM, and enabling robust disaster recovery all become  complex tasks, often forcing teams to rely on custom scripts or inefficient manual processes.

Introducing cross-project backup and restore

To address this, Backup for GKE, now in preview, supports cross-project backup and restore. This new capability allows you to back up workloads from a GKE cluster in one Google Cloud project, securely store the backups in a second, and restore them to a cluster in a third. This streamlines data protection, enhances your security posture, and offers greater flexibility for your operational workflows.Storing backups in a separate, isolated project and region is essential for modern disaster recovery, safeguarding your recovery capability during a regional outage or a compromise in a primary Google Cloud project — the foundation of a resilient infrastructure. This separation also simplifies regulatory compliance, boosts security by limiting the blast radius of any potential incident, and helps you meet RTO/RPO objectives.

Key benefits of cross-project backup and restore 

• Centralized backup management: Consolidate GKE backups from multiple Google Cloud projects into a single project by pointing the backup plan for each cluster to the chosen backup project. This simple configuration provides your team with one control plane to oversee monitoring and manage backup policies.

• Enhanced disaster recovery: Storing GKE backups in a separate project and region provides a vital layer of isolation, boosting your resilience against events like regional outages. If your source region becomes unavailable, you can create a restore plan from your backup project to recover your workloads to a cluster in another project.

• Streamline operations: seeding, cloning, and collaboration

  Cross-project capabilities bring agility to your development lifecycle by simplifying how you copy data between environments. You can now leverage production backup data for testing or rapidly clone entire application environments.

• • Seed and clone environments: You can populate a staging environment with data from a prior backup or create a sandbox. Create a restore plan using an existing backup plan located in the backup project, then select a backup — such as one from production for seeding or a dev environment for cloning — and target a cluster in any other project as your destination. This lets you create test environments and isolated sandboxes.

  • Simplify cross-team collaboration: Since all backups are stored in a central backup project, you can grant a developer from another team a role like Delegated Restore Admin, and also provide them with read permission on the specific backup plan and all of its associated backups. They can then use it to restore to their cluster without needing access to the other team’s live source project.

• Achieve separation of duties for security and compliance

  Isolating backups in a dedicated project allows you to enforce the principle of least privilege by assigning distinct responsibilities. You can empower your application teams with self-service permissions to back up and restore applications within their own projects, without giving them control over the central backup repository. A central platform or operations team can be granted administrative control over the backup project to govern the entire data lifecycle — from setting retention policies with immutability to conducting audits, all without needing access to live production environments. This separation is key to reducing risk and simplifying audits.

  For detailed guidance on Backup for GKE IAM roles and permissions, see the documentation.

Cross-project backup and restore for GKE helps you protect your containerized workloads across multiple Google Cloud projects. This feature allows you to strengthen your disaster recovery capabilities, improve your security posture, and streamline operational workflows.

Get started today

Want to try this feature yourself? To enable it for your projects, please complete this form.

• Learn how to perform cross-project backups

• Learn how to perform cross-project restores

In the fast-paced world of AI, it can be challenging to pause and reflect on how we work. Yet this reflection is the cornerstone of continuous improvement. The 2025 DORA survey offers a unique opportunity for you and your team to do just that. By taking just 10-15 minutes to participate before July 18, you can gain valuable insights into your team’s current practices and identify immediate opportunities for growth.

The benefits of applying DORA principles are not just theoretical. Companies around the world have seen significant improvements in their software delivery and operational performance. For example, financial services giant Banorte was able to increase their deployment frequency from bi-weekly to multiple times a day. Similarly, by focusing on continuous improvement, SLB cut their deployment time from five days to just three hours.

These are not isolated incidents. GitLab reduced errors by an impressive 88%, enhancing platform stability, while Scoops accelerated their feature delivery by 700%. These organizations, and many others, have demonstrated that a commitment to getting better at getting better, supported by the insights from DORA, can lead to transformative results.

The 2025 DORA Survey is available in multiple languages, including English, French, Japanese, Portuguese, Simplified Chinese, and Spanish, to ensure that we can capture a truly global snapshot of the technology landscape. Your participation, regardless of your location or language, is invaluable.

Charting the future of AI in technology teams

DORA’s research has already shown that AI is transforming every stage of the software development lifecycle, with 76% of technologists reporting they rely on AI in their daily work. Our findings indicate that when organizations are transparent about their AI strategy and create policies to govern its adoption, they see a significant increase in the use of these powerful new tools. We’ve also seen that AI can have a positive impact on developer well-being, leading to higher job satisfaction and less burnout, but that organizations must be mindful of how AI affects the perceived value of development work.

This year’s DORA survey will take an even deeper look into how we can all improve the impact of AI across the entire software lifecycle. We’ll be building on our previous research into the importance of fostering trust in AI and how to best support teams as they adopt these new capabilities. 

What’s in it for you?

The act of taking the DORA survey is, in itself, a valuable team-assessment. It provides a structured way to think about your team’s workflows, from initial idea to end-user delivery. This reflection can spark immediate ideas for improvement. Perhaps you’ll identify a bottleneck in your review process or an opportunity to enhance your documentation. High-quality documentation, for instance, has been shown to substantially improve team performance.

Your participation also contributes to a larger body of research that helps the entire technology community understand what it takes to build and maintain high-performing teams. By sharing your anonymous experiences, you help shape the industry’s understanding of the impact of AI and the characteristics of high-performing teams. This research is strongest when it includes diverse perspectives, which is why we are calling on a wide range of roles to participate. Whether you are a software engineer, a product manager, a CISO, or a UX designer, or anyone else who participates in the creation and delivery of software, your voice is critical.

Take the DORA survey today

Ready to take the first step towards improving your team’s performance? The 2025 DORA Survey is open until July 18, 2025. Take 15 minutes to reflect on your team’s practices, contribute to industry-leading research, and discover how you can get better at getting better.