gcp

In the world of database management, understanding performance bottlenecks is critical to smooth operations and an optimal user experience. Using managed database services can help alleviate mundane management tasks and let you focus on value-added, strategic tasks, while also offering tools for monitoring database and resource performance. But when performance issues arise, gathering and analyzing various system and database metrics before and during the incident to conduct a thorough analysis can be time-consuming and labor-intensive. 

Google Cloud’s AlloyDB for PostgreSQL is a fully managed database service that delivers superior availability, scalability, and performance. Recently, we added a new feature called performance snapshot report that provides deep insights into database performance. In this blog post, we explore how the performance snapshot report lets you identify and solve performance issues with greater clarity, control, and ease. This tool complements other AlloyDB observability features like systems insights, query insights, and the Metrics Explorer, which provide real-time metrics about your instance.

Demystifying the performance snapshot report

The performance snapshot report is a collection of PostgreSQL functions and views that captures snapshots of your AlloyDB system statistics. These snapshots provide detailed diff reports of performance metrics between different points in time, offering a comprehensive view of your database’s activity during the period.

Here’s how it works

The performance snapshot report captures snapshots of your database’s performance metrics at any point during workload execution and generates reports based on any two snapshots. This gives you granular control over what periods you want to analyze, whether it’s the entire workload, specific intervals, or moments of peak activity. Each snapshot captures a wealth of information, including changes in key performance indicators like wait events, I/O statistics, and query execution times. It also captures system configuration details such as CPU usage, memory allocation, and parameter details.

You can also access and analyze performance by generating reports between two snapshots directly within your PostgreSQL environment, using familiar psql queries, as in familiar Oracle AWR reports, facilitating easy adoption and integration into existing processes. Then, you can save the reports for historical comparisons to identify regressions, evaluate changes, and monitor performance trends. 

Performance snapshot report in action

The performance snapshot report equips you with the tools you need to diagnose and resolve performance issues effectively. Let’s explore some common scenarios where a performance snapshot report proves to be invaluable.

1. Identifying performance bottlenecks

Scenario: The Cymbal Shop’s database, managed by DBA Sarah, is experiencing an unexpected performance degradation. A previously stable system became significantly slower this morning. Sarah needs to investigate this sudden slowness to determine its cause and implement a solution to address the changes that led to it.

Solution: Sarah had already established a baseline by creating a report comparing two performance snapshots taken two days prior during normal system operation. To investigate the unexpected slowdown, Sarah first uses the `perfsnap.snap()` function to capture performance snapshots at two distinct time points, separated by approximately one hour. Subsequently, she utilizes the `perfsnap.report(start_snap_id, end_snap_id)` function to generate a differential report, which highlights variations in critical performance indicators such as wait events, I/O activity, and query execution times. Finally, Sarah analyzes the generated report, focusing on significant increases in particular wait events — such as `lwlock` for lock contention or `io` for storage delays — and changes in query execution durations. This detailed examination allows her to precisely identify the root cause of the sudden performance degradation.

Example:

That generates a report similar to this example report.

2. Assessing performance impact post system upgrades and configuration changes

Scenario: Sarah has upgraded the AlloyDB version. She’s also modified a few configuration parameters. She wants to understand the system behavior post these changes.

Solution:

1. Make baseline snapshot report: Before making the changes, Sarah captured two snapshots to create a report that shows the baseline stats.

2. Create post-change snapshot report: After making the changes, she captured two snapshots, and generated another report. 

3. Analyze changes: She compared the report for any significant changes in performance metrics by comparing the two reports. Pay attention to sections like “CPU Utilization,” “Memory Utilization,” and “I/O Statistics” to assess the impact of the changes.

Know thy database performance

The performance snapshot report is a convenient way for you to get visibility and control over your AlloyDB database performance. By capturing and comparing snapshot reports, you can identify bottlenecks, optimize workloads, assess changes, and diagnose performance regressions with ease. This powerful tool, combined with the inherent performance advantages of AlloyDB, empowers you to unlock the full potential of your database and ensure your applications run smoothly and efficiently. 

If you’re new to Google Cloud and want to take AlloyDB for a spin, just sign up for an AlloyDB free trial. If you’re already a Google Cloud user, head over to the AlloyDB console, click “Create a trial cluster” and we’ll guide you through migrating your PostgreSQL data to a new AlloyDB database instance.

Learn more 

• Review the documentation for the performance snapshots rReport

• Watch this session for a deep dive into the latest innovations in AlloyDB

• Learn more about AlloyDB

Editor’s note: The Jina AI Reader is a specialized tool that transforms raw web content from URLs or local files into a clean, structured, and LLM-friendly format.  In this post, Han Xiao details how Cloud Run empowers Jina AI to build a secure, reliable, and massively scalable web scraping system that remains economically viable. This post explores the collaborative innovation, technical hurdles, and breakthrough achievements behind Jina Reader, a web grounding system now processing 100 billion tokens daily.

When Jina Reader launched in April 2024, its explosive growth — serving more than 10 million requests and 100 billion tokens daily — confirmed huge demand for reliable, LLM-friendly web content. Jina Reader isn’t just another scraper; it takes a different approach to  how AI systems consume web content by transforming raw, noisy web pages into clean, structured markdown.

The core challenge for any AI system processing web data is the “web grounding problem.” Modern websites are a chaotic mix of content, ads, tracking scripts, and dynamic JavaScript, creating an overwhelming noise-to-signal ratio. Traditional scrapers struggle with this complexity, often failing on dynamic single-page applications or generating unusable, ungrounded data for LLMs. Jina Reader’s breakthrough, ReaderLM-v2, is a purpose-built 1.5-billion-parameter language model that intelligently extracts content, trained on millions of documents to understand web structure beyond simple rules.

Cloud Run: The engine behind Jina Reader’s scale

Jina Reader faced  inherent burstiness and unpredictability of web scraping workloads. Traditional virtual machine setups meant either costly over-provisioning or critical failures under load. Google Cloud Run became the essential solution, enabling Jina Reader to build a web scraping system that is secure, reliable, massively scalable, and economically viable.

• The web grounding app (the browser automation system that scrapes and cleans web content) is hosted on Cloud Run (CPU). It runs full Chrome browser instances.

• ReaderLM-v2 is a purpose-built 1.5-billion-parameter language model for HTML-to-markdown conversion that runs on Cloud Run with serverless GPUs.

Cloud Run directly addressed several critical issues:

• Optimized Performance: The deep collaboration between Jina Reader and Google Cloud engineering was essential. We jointly optimized container lifecycle management for browser automation, reducing startup times from over 10 seconds to under two seconds  through prewarming, optimized images, and intelligent resource allocation. For ReaderLM-v2, Google’s team helped create custom container configurations to efficiently run a 1.5-billion-parameter model on Cloud Run GPUs. The on-demand scaling and fast start capabilities of Cloud Run GPUs were critical in helping optimize model performance, directly impacting our ability to process 100 billion tokens daily.

• True Scale-to-Zero Serverless: Cloud Run’s ability to run full Chrome browser instances allowed cost-effective operations. Each request spawns an isolated container with its own headless Chrome, and crucially, these containers disappear when the request is done. This ephemeral nature is vital for processing untrusted web content, mitigating security risks and memory leaks.

• Global Multi-Regional Deployment: Cloud Run’s global presence ensures requests are processed close to both the users and target websites. This significantly minimizes latency and boosts success rates, even against geo-restricted content.

• Massive & Automatic Scaling: The platform seamlessly scales from a handful to over 1,000 container instances during peak traffic, handling the unpredictable nature of web scraping without manual intervention.

• Economic Viability: With Cloud Run’s pay-per-use model, Jina Reader can offer a generous free tier to end users while maintaining profitability even with substantial monthly usage. This pricing flexibility was fundamental to our widespread adoption.

• Resilience and Operational Excellence: During a recent sustained DDoS attack, Cloud Run’s serverless architecture proved invaluable. It scaled up to absorb massive loads (over 100,000 requests per minute), while intelligent rate limiting filtered malicious traffic. Critically, costs returned to normal immediately after the attack subsided due to its scale-to-zero capability.  The system has maintained over 99.9% uptime.

Conclusion

Building Jina Reader on Google Cloud Run proved that AI capabilities and cloud-native architecture are complementary. Cloud Run’s unique capabilities — serverless GPUs, container isolation, global deployment and scale-to-zero economics — made the architecture possible. Our close partnership demonstrates that deep integration between AI-first systems and modern cloud infrastructure can create capabilities previously thought impossible, enabling us to process 100 billion tokens every day.

You can discover more about Cloud Run GPUs on our product page, and if you want to learn how to host a large language model on Cloud Run, watch this video.

As one of India’s largest healthcare providers, Manipal Hospitals serves nearly 7 million patients annually across 37 hospitals. To deliver clinical excellence and patient-centric care at a high standard, we are continually embracing technology.  One of our most significant operational challenges we consistently face is the nurse handover process—a critical but time-consuming task. To make nurse handovers more efficient, safe and accurate, we entered a strategic partnership with the Google Cloud Consulting team to co-develop a generative AI solution, leveraging the power of  Google Cloud.

Rethinking time-consuming, error-prone nurse handoffs

The process of transferring essential information about a patient’s condition and care plan from an outgoing nurse to an incoming one is crucial for ensuring continuity of care and patient safety. However, with more than 10,500 beds across our hospitals, the sheer volume of data required for a comprehensive handover meant our nurses routinely added an extra 90 minutes to their shifts for both creating and receiving these reports. This lengthy process could directly affect patient care, as it could lead  to fatigue and potential mistakes, and also reduce job satisfaction for our vital nursing staff. We needed a way to make this process faster, more accurate, and less of a burden.

Building a trusted solution on Google Cloud

Our joint Manipal-Google team knew that for a clinical tool to be adopted by over 5,000 nurses, it had to be both fast and trustworthy. The primary challenge with any generative AI application in healthcare is ensuring accuracy and minimizing the risk of AI “hallucinations.”

The solution’s architecture, designed by the Google Cloud Consulting team, addresses this head-on by leveraging multiple Google Cloud components. Patient data from our TrakCare system is securely transferred in near real-time to a data lake on Google Cloud. When a nurse requests a handover summary, a serverless Cloud Run application orchestrates a multi-stage process.

Critically, instead of passing pages of raw data directly to the AI, the system first uses intelligent, time-based filters to extract only the most relevant clinical information for the specific shift. This structured, pre-processed data is then sent to Gemini on Vertex AI. This “controlled generation” approach was a key innovation; it ensures Gemini summarizes only the most pertinent facts, dramatically improving the accuracy and consistency of the final ISBAR (Identify, Situation, Background, Assessment, and Recommendation) report. The ability of Gemini to understand complex medical terminology, medication names, and clinical procedures without specialized fine-tuning was a game-changer, accelerating the entire development process.

How our partnership delivered results

By combining Manipal’s deep clinical expertise and Google Cloud Consulting’s technical leadership, our joint approach provides a blueprint for enterprise-grade AI implementation:

• From ideation to production: The Google Cloud Consulting team led the engagement from the initial idea all the way to a production-ready solution now used by thousands of nurses daily. The project started with a focused Minimum Viable Product (MVP) to prove the technology’s value before scaling.

• User-centric design: The solution was not built in a vacuum. The Google team conducted over eight rounds of deep discussion and evaluation sessions directly with our nurses. This ensured the final ISBAR summary format was not just technically impressive, but clinically useful from day one.

• Agile and iterative rollout: The solution was piloted at one hospital initially to test its performance and safety in a real-world setting. With a successful pilot, the solution is live in 23 of Manipal hospitals, and used by more than 5,000 every day. At full scale, it is projected to help save significant nurse hours on a daily basis.  This phased approach, managed jointly, has allowed us to gather feedback and ensure smooth adoption.

Ensuring better patient care

The generative AI solution we implemented has yielded impressive returns. The 70% reduction in handoff time—from 90 minutes down to 20—frees our nurses to focus more on direct patient needs and care. It also makes the process less vulnerable to errors that can arise from handwritten notes and human fatigue.

This project, delivered in partnership with Google Cloud Consulting, is a prime example of how we are pioneering the future of healthcare in India, helping us scale the delivery of quality care across the length and breadth of the country.

^{We’d like to give special thanks to Google Cloud Consulting team –  Naveen Poosarla, Gopala Dhar, Rupjit Chakraborty, Hem Anand, Amit Dutta, Nishant Welpulwar, Preetam Dey and Shikha Saxena – for designing and developing the solution. We are grateful to the Manipal Hospitals team – Saroja Jaykumar, Sunil Bhattacharjee –  in delivering this successful project.}

As Google Kubernetes Engine (GKE) deployments grow and scale, adopting a multi-project strategy in Google Cloud becomes a best practice for security and environment organization. Creating clear boundaries by using distinct projects for development, testing, and production environments provides isolation and helps manage access control.

However, isolation introduces a data protection challenge: How do you effectively manage backups across these project boundaries? Without a native solution, centralizing backups, ensuring a clear separation of duties with IAM, and enabling robust disaster recovery all become  complex tasks, often forcing teams to rely on custom scripts or inefficient manual processes.

Introducing cross-project backup and restore

To address this, Backup for GKE, now in preview, supports cross-project backup and restore. This new capability allows you to back up workloads from a GKE cluster in one Google Cloud project, securely store the backups in a second, and restore them to a cluster in a third. This streamlines data protection, enhances your security posture, and offers greater flexibility for your operational workflows.Storing backups in a separate, isolated project and region is essential for modern disaster recovery, safeguarding your recovery capability during a regional outage or a compromise in a primary Google Cloud project — the foundation of a resilient infrastructure. This separation also simplifies regulatory compliance, boosts security by limiting the blast radius of any potential incident, and helps you meet RTO/RPO objectives.

Key benefits of cross-project backup and restore 

• Centralized backup management: Consolidate GKE backups from multiple Google Cloud projects into a single project by pointing the backup plan for each cluster to the chosen backup project. This simple configuration provides your team with one control plane to oversee monitoring and manage backup policies.

• Enhanced disaster recovery: Storing GKE backups in a separate project and region provides a vital layer of isolation, boosting your resilience against events like regional outages. If your source region becomes unavailable, you can create a restore plan from your backup project to recover your workloads to a cluster in another project.

• Streamline operations: seeding, cloning, and collaboration

  Cross-project capabilities bring agility to your development lifecycle by simplifying how you copy data between environments. You can now leverage production backup data for testing or rapidly clone entire application environments.

• • Seed and clone environments: You can populate a staging environment with data from a prior backup or create a sandbox. Create a restore plan using an existing backup plan located in the backup project, then select a backup — such as one from production for seeding or a dev environment for cloning — and target a cluster in any other project as your destination. This lets you create test environments and isolated sandboxes.

  • Simplify cross-team collaboration: Since all backups are stored in a central backup project, you can grant a developer from another team a role like Delegated Restore Admin, and also provide them with read permission on the specific backup plan and all of its associated backups. They can then use it to restore to their cluster without needing access to the other team’s live source project.

• Achieve separation of duties for security and compliance

  Isolating backups in a dedicated project allows you to enforce the principle of least privilege by assigning distinct responsibilities. You can empower your application teams with self-service permissions to back up and restore applications within their own projects, without giving them control over the central backup repository. A central platform or operations team can be granted administrative control over the backup project to govern the entire data lifecycle — from setting retention policies with immutability to conducting audits, all without needing access to live production environments. This separation is key to reducing risk and simplifying audits.

  For detailed guidance on Backup for GKE IAM roles and permissions, see the documentation.

Cross-project backup and restore for GKE helps you protect your containerized workloads across multiple Google Cloud projects. This feature allows you to strengthen your disaster recovery capabilities, improve your security posture, and streamline operational workflows.

Get started today

Want to try this feature yourself? To enable it for your projects, please complete this form.

• Learn how to perform cross-project backups

• Learn how to perform cross-project restores

In the fast-paced world of AI, it can be challenging to pause and reflect on how we work. Yet this reflection is the cornerstone of continuous improvement. The 2025 DORA survey offers a unique opportunity for you and your team to do just that. By taking just 10-15 minutes to participate before July 18, you can gain valuable insights into your team’s current practices and identify immediate opportunities for growth.

The benefits of applying DORA principles are not just theoretical. Companies around the world have seen significant improvements in their software delivery and operational performance. For example, financial services giant Banorte was able to increase their deployment frequency from bi-weekly to multiple times a day. Similarly, by focusing on continuous improvement, SLB cut their deployment time from five days to just three hours.

These are not isolated incidents. GitLab reduced errors by an impressive 88%, enhancing platform stability, while Scoops accelerated their feature delivery by 700%. These organizations, and many others, have demonstrated that a commitment to getting better at getting better, supported by the insights from DORA, can lead to transformative results.

The 2025 DORA Survey is available in multiple languages, including English, French, Japanese, Portuguese, Simplified Chinese, and Spanish, to ensure that we can capture a truly global snapshot of the technology landscape. Your participation, regardless of your location or language, is invaluable.

Charting the future of AI in technology teams

DORA’s research has already shown that AI is transforming every stage of the software development lifecycle, with 76% of technologists reporting they rely on AI in their daily work. Our findings indicate that when organizations are transparent about their AI strategy and create policies to govern its adoption, they see a significant increase in the use of these powerful new tools. We’ve also seen that AI can have a positive impact on developer well-being, leading to higher job satisfaction and less burnout, but that organizations must be mindful of how AI affects the perceived value of development work.

This year’s DORA survey will take an even deeper look into how we can all improve the impact of AI across the entire software lifecycle. We’ll be building on our previous research into the importance of fostering trust in AI and how to best support teams as they adopt these new capabilities. 

What’s in it for you?

The act of taking the DORA survey is, in itself, a valuable team-assessment. It provides a structured way to think about your team’s workflows, from initial idea to end-user delivery. This reflection can spark immediate ideas for improvement. Perhaps you’ll identify a bottleneck in your review process or an opportunity to enhance your documentation. High-quality documentation, for instance, has been shown to substantially improve team performance.

Your participation also contributes to a larger body of research that helps the entire technology community understand what it takes to build and maintain high-performing teams. By sharing your anonymous experiences, you help shape the industry’s understanding of the impact of AI and the characteristics of high-performing teams. This research is strongest when it includes diverse perspectives, which is why we are calling on a wide range of roles to participate. Whether you are a software engineer, a product manager, a CISO, or a UX designer, or anyone else who participates in the creation and delivery of software, your voice is critical.

Take the DORA survey today

Ready to take the first step towards improving your team’s performance? The 2025 DORA Survey is open until July 18, 2025. Take 15 minutes to reflect on your team’s practices, contribute to industry-leading research, and discover how you can get better at getting better.

SQL has been the cornerstone of data management for over 50 years, valued for its declarative nature and robust ecosystem. However, traditional SQL has limitations, including rigid clause structures, verbose syntax, and complex nested queries that can hinder readability and maintenance. To address these challenges, Google introduced pipe syntax, an extension to GoogleSQL that reimagines how queries are written and processed. A pipe-structured data flow makes SQL more easier to read and write than ever before.

As we discussed in a previous blog post, pipe syntax introduces a linear, top-down approach to SQL queries, using the pipe operator (|>) to chain operations like filtering, aggregating, and joining in a logical sequence. This structure aligns with the natural flow of data transformations, making queries more intuitive, readable, and maintainable. Unlike standard SQL, which enforces a strict order (SELECT, FROM, WHERE, etc.), pipe syntax allows operations to be applied in any order, reducing the need for cumbersome subqueries or Common Table Expressions (CTEs). 

Since pipe syntax became generally available in April, we have seen you embrace it across a myriad of use cases — from streamlining data transformations, building insightful reports to efficiently analyzing logs, and many more. In this blog, we are highlighting three different use cases from customers about how they are using pipe syntax to simplify data transformations and log data analysis.

1.Supercharging data analysis with a linear data flow 

Imagine you want to build a recommendation report to suggest whether a logical or physical pricing model is more cost effective for long-term data storage in BigQuery. While physical storage is generally cheaper, costs can vary based on factors like data compressibility.

You might write a query like this using standard SQL:

Using pipe syntax, the query can be written as:

Pipe syntax eliminates the need for CTEs, allowing users to chain operations linearly. This reduces the query’s complexity and makes it easier to follow, especially for cost analysis tasks requiring multiple transformations. Now you can build a report faster, and share the queries across teams that are easier for them to understand. 

2. Simplifying data transformations in data pipelines 

Suppose you are building a data pipeline for a dashboard. You want to process customer order data to calculate total revenue per customer, filter high-value customers, and then rank them.

In standard SQL:

Using pipe syntax, the query can be written as:

Building a data pipeline is a multi-step process involving data cleaning, transformation and aggregation. Pipe syntax simplifies developers’ workflows by allowing them to chain operations like aggregation and filtering in a logical and sequential way. The simplicity of the code allows developers to focus more time on solving business problems instead of writing and debugging cumbersome SQL code. Fewer lines of code and higher productivity eventually translate to faster business outcomes. 

3. Simpler log analysis in Google Cloud Logging 

Log Analytics in Google Cloud Logging is a powerful tool for storing, searching, analyzing, and monitoring application and system log data through SQL. When you need to perform complex analysis, aggregations, or transformations on their logs, the Log Analytics feature gives you the full strength of SQL.

When helping enterprises to migrate their workloads and logging tooling to Cloud Logging, translating pipe-like structure into standard SQL queries for typical log analysis tasks can sometimes feel cumbersome; it involves subqueries and common table expressions (CTEs) that might not feel immediately intuitive. Pipe syntax offers a more linear, pipe-based syntax, often making common log analysis patterns easier to read and write.

Let’s look at a concrete example.

The challenge: Analyzing log frequency by time, severity, and resource attribute

Imagine you want to analyze your logs to find patterns where specific log messages (identified by severity and tags) occur frequently (more than 5 times per minute), excluding certain messages (like those containing “CookieIncluded=False”).

Using standard SQL in Log Analytics, you might write a query like this:

Using pipe syntax, the query can be written as:

Beyond looking different, pipe syntax fundamentally changes how you build and read queries for sequential log processing. It tackles two common pain points of standard SQL: nested subqueries and “inside-out” logic.

PipeSQL eliminates this nesting for common sequential tasks. Each pipe (|>) feeds the previous command’s output into the next. Aggregation results from the summary are immediately available for filtering in the subsequent WHERE clause without needing an explicit nested structure. The query remains flat and linear. By tackling nesting and promoting a linear flow, pipe syntax significantly lowers the cognitive load for many common log analysis tasks, making powerful analytics more accessible.

Customer love for pipe syntax 

“I was very excited when BigQuery first introduced pipe syntax. As someone who is familiar with R, it is very natural for myself and our team of analysts to use pipe-like syntax for more streamlined data transformation via Dataform. I am happy to share that I have seen >30% reduction in my code compared to standard SQL. This eventually transferred to time saving to read, write and debug queries. I am looking forward to onboarding more analysts to use pipe syntax across my organization.” – Shanker Venkatachalam, Senior Manager, Business Strategy, Northwell.edu

“The pipe syntax has proven to be a revelation in my analytical work. Although I initially had concerns about the learning curve, it was surprisingly smooth. The linear flow facilitates step-by-step query construction, ensuring each transformation is correct before moving on. This reduces complexity and improves long-term maintainability. Notably, it also naturally encourages a ‘micro-transformations’ mindset, resulting in significantly more organized and understandable queries.”– Axel Thevenot, Google Developer Expert, Venom Engineering

“At Bindplane, we specialize in accelerating migrations to Cloud Observability with our telemetry pipeline built on OpenTelemetry. Once the data arrives, using pipe syntax makes it much easier to build key observability assets like dashboards, alerts, and queries. Once pipe syntax is available, I’m done writing regular SQL. It’s just so much easier for helping enterprises get up and running with Cloud Logging/Log Analytics.” – Keith Schmitt, Senior Software Engineer, Bindplane 

What’s next for pipe syntax?

As shared at Cloud Next, BigQuery supports Gemini-powered Data Preparation Agents, which can analyze your data and auto-generate data pipelines to clean and transform your data. You will soon be able to use pipe queries to validate generated queries, which simplifies your pipelines and makes it easier to review and verify code.

In short, pipe syntax is more than a syntactic upgrade. It’s a paradigm shift that makes SQL more accessible and productive for analysts, engineers, and data teams. Its modular, pipeline-based approach aligns with modern data workflows, saving development time and improving efficiency. Whether you’re building reports, developing data pipelines, or analyzing logs, pipe syntax delivers shorter, clearer, and more efficient queries, empowering you to focus on insights rather than syntax. 

Ready to simplify your SQL workflows? Learn how to use pipe syntax from this tutorial, and check out the following resources: 

• Pipe syntax documentation and detailed reference guide 

• Demo video, tutorial 

• Blog post on using pipe syntax for log data analysis

• Research publication (VLDB 2024) with more background and language design details

• Open-source GoogleSQL implementation, ZetaSQL

As your operational needs change, sometimes you need to move data residing within Google’s Cloud Storage to a new location, to improve resilience, optimize performance, meet compliance needs, or simply to reorganize your infrastructure. Yet moving buckets can be a daunting, complex, risky endeavor that involves manual scripting, painstaking coordination, and the risk of data loss, or worse yet, extended downtime. This can discourage organizations from making the changes they need to their storage environments.

We recently introduced Cloud Storage bucket relocation, a unique feature among leading hyperscalers that makes it easy  to change your bucket’s location. Bucket relocation eliminates the need for complex manual planning and helps prevent extended downtime, for an easy transition with minimal application disruption, and strong data integrity. Your bucket’s name, and all the object metadata within it, remain identical throughout the relocation, so there are no path changes, and your applications experience minimal downtime while the underlying storage is moved. Furthermore, your objects retain their original storage class (e.g., Standard, Nearline, Coldline, Archive) and time-in-class in the new location. This is key for many cost efficiency strategies, helping ensure capabilities such as Autoclass continue to operate intelligently to optimize your storage costs post-migration.

Bucket relocation is a key capability within the Storage Intelligence suite, alongside tools like Storage Insights, which  provides deep visibility into your storage landscape and identifies optimization opportunities. Bucket relocation then lets you act on these insights, and move your data between diverse Cloud Storage locations — regional locations for low latency, dual-regions for high availability and disaster recovery, or multi-regions for global accessibility — to meet your  business, performance, and compliance objectives.

Bucket relocation under the hood

Bucket relocation relies on two critical techniques. 

• Asynchronous data copy: Bucket relocation leverages a unique and optimized asynchronous data transfer mechanism that copies data in the background to  minimize impact to ongoing operations. Existing operations like writing, reading, and updating objects continue while the entire dataset is being copied.

• Metadata preservation: Historically, Google Cloud customers moved data with the Storage Transfer Service, which copied the objects to a new bucket and deleted existing ones. Bucket relocation, on the other hand, automatically and meticulously moves all your bucket’s and objects’ associated metadata, thereby preserving state. This includes information like:

• Storage class: Your objects retain their original storage class (e.g., Standard, Nearline, Coldline, Archive) in the new location.

• Bucket and object names: The naming structure of your buckets and objects remains identical.

• Creation and update timestamps: These markers are preserved, so that features like object lifecycle management (OLM) rules continue to operate.

• Access Control Lists (ACLs) and IAM policies: Bucket- and object-level permissions are transferred to help maintain your security posture.

• Custom metadata: Any user-defined metadata associated with your objects is also migrated.

By handling the complexities of asynchronous data transfer and automatic metadata migration, bucket relocation minimizes the risks and overhead associated with a manual bucket migration. Crucially, because the bucket name is preserved throughout the relocation process, applications accessing the bucket don’t need to be modified.

Relocate your bucket in a few simple steps

With bucket relocation, you can move your Cloud Storage buckets in three simple steps. Here’s a breakdown:

1. Initiate a dry run:

• Before starting the actual relocation, it’s highly recommended to perform a dry run. This simulates the process without moving any data, allowing you to identify potential issues early on, such as incompatible configurations.
• The dry run checks for incompatibilities like customer-managed encryption keys (CMEK), locked retention policies, objects with temporary holds, and bucket tags, without you having to manually validate each of them.
• Make sure to add the --dry-run flag!

Replace BUCKET_NAME with the name of your bucket and LOCATION with the desired destination.

2. Start the relocation process:

• This step initiates the actual data transfer from the source bucket to the destination bucket. During this phase, you can still read, modify, and delete objects in the bucket. However, the bucket metadata (i.e., bucket-level  parameters and configurations) is write-locked to prevent changes that could affect the relocation.
  Note: Removing the --dry-run flag from the dry-run command initiates the relocation.

3. Finalize the relocation process:

• Once the incremental data copy is complete, you’re ready to trigger the final synchronization step (except when moving between multi-region and configurable dual-region). This involves a brief period where writes to the bucket are disabled to help ensure their data integrity; any last-second changes made to the objects within the bucket while the incremental copy was in progress are copied to the destination. After the data’s integrity is verified, the bucket’s location is updated, and all requests are automatically redirected to the new location. During the final synchronization step, attempts to update objects in the bucket will result in an HTTP 412 error.
• Do not initiate the final synchronization process until the relocation process progress reaches ~99%. This helps you minimize downtime because most of the data has already been synchronized in the background. 

Note: If you’re moving between multi-regions and configurable dual-regions within the same multi-region code, you’re all set — bucket relocation handles the transition in the background, no finalization or downtime required!

The OPERATION_ID is provided as output from Step-2. The OPERATION_ID is listed with the keyword name. For instance:

name: projects/_/buckets/my-bucket/operations/AbCJYd8jKT1n-Ciw1LCNXIcubwvij_TdqO-ZFjuF2YntK0r74

And there you have it — In just three steps, you’ven moved your entire bucket, its data, and metadata, to its new location.

Early users of bucket relocation have had great success with the new feature. 

“With Storage Intelligence and bucket relocation, we effortlessly transitioned to dual-region buckets. The seamless process, powered by the bucket relocation, minimized downtime and ensured data integrity. We migrated the buckets with peace of mind and without the manual headaches.” – Adam Steele, Product Manager, Spotify

“We recently utilized the bucket relocation feature of Storage Intelligence to successfully complete a ~300 bucket migration and PBs of data project from multi-region to regional storage, to optimize network data transfer costs. Without bucket relocation, this process would have required extensive automation and scripting, resulting in increased downtime and effort.” – Deepak Mahato, Data Platform Infrastructure Manager, GroupOn

Experience the ease and efficiency of managing your Cloud Storage buckets with bucket relocation in Storage Intelligence. To learn more, visit the bucket relocation documentation and the Storage Intelligence overview.

In today’s hyperconnected world, delivering personalized content at scale requires more than just aggregating information – it demands deep understanding of context, relationships, and user preferences. Glance, a leading content discovery platform that delivers personalized, real-time content experiences on mobile lock screens across the globe, serves over 300 million users worldwide across 450 million devices. Beyond news aggregation, Glance curates diverse content including entertainment, sports, gaming, shopping, and lifestyle content, making every glance at the phone screen meaningful and engaging.

However, with the exponential growth of digital content, particularly the overwhelming volume of daily news articles, Glance faced a critical challenge: how to effectively navigate this information while maintaining the personalized, contextual experiences users expect. The existing search and content discovery capabilities needed significant enhancement to uncover emerging trends, improve search relevance, provide deeper context, and most importantly, deliver truly personalized content recommendations that resonate with individual user preferences and behaviors.

Glance partnered with Google Cloud Consulting team to build a sophisticated Content Knowledge Graph (CKG) that addresses these challenges head-on. This solution leverages Google Cloud’s advanced AI and data processing capabilities, including Gemini models, BigQuery, Vertex AI, and Google Cloud partner Neo4j, to ingest, process, extract, standardize, classify, and structure news data into a dynamic network of interconnected entities and relationships. The Content Knowledge Graph has dramatically improved search relevance, enhanced personalized content discovery, provided deeper contextual insights, increased user engagement, and improved scalability and efficiency.

Tackling the content discovery and personalization challenge  

Glance’s mission extends far beyond traditional content aggregation. As a platform serving hundreds of millions of users globally, Glance must deliver hyper-personalized content experiences that anticipate user interests and adapt to evolving preferences in real-time. The challenge was multifaceted:

• Uncovering trends at scale – Identifying emerging topics, viral content, and the complex relationships between different content categories across news, entertainment, sports, and lifestyle domains

• Enhancing personalized search – Improving the accuracy and relevance of search results based on individual user behavior patterns, preferences, and contextual signals

• Providing intelligent context – Offering users deeper understanding not just of individual pieces of content, but how different stories, events, and topics connect across the broader content ecosystem

• Scaling personalization – Delivering these sophisticated experiences across 300+ million users while maintaining real-time responsiveness and relevance

Manually analyzing and connecting the dots within this vast, multi-dimensional content landscape was simply not scalable. Glance needed an intelligent, automated approach that could understand content at both granular and contextual levels while powering personalized recommendations at unprecedented scale.

Building a Gemini-powered content intelligence engine

Glance and Google Cloud Consulting team collaborated to architect and implement a comprehensive Content Knowledge Graph that transforms how content is understood, connected, and delivered. This sophisticated system leverages the full spectrum of Google Cloud’s AI and data processing capabilities:

• Intelligent content ingestion and processing – The system ingests content from diverse sources beyond news, including entertainment articles, sports updates, lifestyle content, and trending topics, storing them in BigQuery for efficient, scalable processing.

• Advanced entity extraction and relationship mapping – Using Gemini foundational models, the system extracts key entities (people, organizations, locations, events, brands) and identifies complex relationships between them across different content categories.

• Entity standardization and knowledge linking – Extracted entities are normalized using Gemini’s advanced language understanding and linked to authoritative knowledge sources like Wikipedia, ensuring consistency and enabling sophisticated cross-content analysis.

• Multi-dimensional content classification – Gemini foundation models classify content into granular categories using the IAB content taxonomy while also identifying sentiment, urgency, and relevance signals for personalization.

• Intelligent content summarization and tagging – Gemini generates contextual tags, compelling short headlines, and category labels, enabling users to quickly grasp content essence while powering recommendation algorithms.

• Dynamic Knowledge Graph construction – The extracted information is structured into a Neo4j graph database, creating a living, breathing network of interconnected entities, topics, relationships, and user interaction patterns.

• Real-time trend analysis and prediction – The system integrates with external trend APIs and analyzes user engagement patterns to identify and predict trending topics, providing actionable insights for content curation.

• Interactive analytics dashboard – NeoDash powers an interactive dashboard for monitoring trending content, analyzing entity relationships, and visualizing content performance across different user segments.

Diagram 2:

Engineering for Global Scale and Performance

Glance enhanced the Content Knowledge Graph solution to handle 50,000+ daily articles across multiple content categories. The engineering team implemented several critical optimizations:

• Event-driven architecture transformation – Migrating to a Kafka-based event-driven architecture with intelligent retries and asynchronous operations resulted in 4x throughput improvement, enabling real-time content processing at massive scale.

• Graph database optimization – Neo4j query optimization and indexing strategies drastically reduced query response times from seconds to milliseconds, enabling real-time content recommendations.

• Kubernetes-native deployment – Moving to managed Google Kubernetes Engine (GKE) with auto-scaling capabilities improved system reliability and resource utilization.

• Strategic performance optimization – Applying the 80-20 principle to focus on high-impact optimizations, coupled with Redis caching and Cloud Spanner for critical data, reduced processing latency by 80% and boosted recommendation coverage from under 60% to over 85%.

• Intelligent load balancing – Implementing smart load distribution across processing pipelines ensured consistent performance even during viral content spikes and peak traffic periods.

 Business impact

The Content Knowledge Graph has delivered measurable improvements across key business metrics:

• Enhanced content discovery performance – CKG-powered content recommendations boosted Cards per Session (CPS) by 24%, directly improving user engagement and platform stickiness.

• Significantly increased user engagement – More relevant and contextually aware content delivery resulted in higher click-through rates and a 5% increase in swiping sessions, particularly in the related content sections.

• Real-time trend intelligence – Users now discover trending topics instantly across news, entertainment, sports, and lifestyle categories, with faster trend detection compared to previous systems.

• Data-driven content strategy – The CKG provides comprehensive, actionable insights into content performance, user preferences, and emerging trends, enabling data-driven editorial and curation decisions.

• Global scalability and efficiency – The cloud-native architecture seamlessly handles Glance’s ever growing global content pool while maintaining cost efficiency and performance.

Shaping the Future of content discovery

The Glance Content Knowledge Graph transforms  raw, unstructured content into a sophisticated, interconnected knowledge network, and has  has empowered Glance to deliver truly engaging content experiences that anticipate user needs.The solution’s success lies not just in its technical sophistication, but in its ability to enhance the human experience of content discovery – making every interaction with Glance more meaningful, relevant, and engaging.

As content continues to proliferate and user expectations for personalization grow, the principles and technologies demonstrated in this project provide a blueprint for the future of intelligent content platforms. We’re excited to see how Glance leverages this powerful foundation to further innovate in personalized content discovery and set new standards for user experience in the digital content ecosystem.

At Google Cloud, we are committed to making it as seamless as possible for you to build and deploy the next generation of AI and agentic applications. Today, we’re thrilled to announce that we are collaborating with Docker to drastically simplify your deployment workflows, enabling you to bring your sophisticated AI applications from local development to Cloud Run with ease. 

Deploy your compose.yaml directly to Cloud Run

Previously, bridging the gap between your development environment and managed platforms like Cloud Run required you to manually translate and configure your infrastructure. Agentic applications that use MCP servers and self-hosted models added additional complexity. 

The open-source Compose Specification is one of the most popular ways for developers to iterate on complex applications in their local environment, and is the basis of Docker Compose. And now, gcloud run compose up brings the simplicity of Docker Compose to Cloud Run, automating this entire process. Now in private preview, you can deploy your existing compose.yaml file to Cloud Run with a single command, including building containers from source and leveraging Cloud Run’s volume mounts for data persistence. 

Supporting the Compose Specification with Cloud Run makes for easy transitions across your local and cloud deployments, where you can keep the same configuration format, ensuring consistency and accelerating your dev cycle.

“We’ve recently evolved Docker Compose to support agentic applications, and we’re excited to see that innovation extend to Google Cloud Run with support for GPU-backed execution. Using Docker and Cloud Run, developers can now iterate locally and deploy intelligent agents to production at scale with a single command. It’s a major step forward in making AI-native development accessible and composable. We’re looking forward to continuing our close collaboration with Google Cloud to simplify how developers build and run the next generation of intelligent applications.” – Tushar Jain, EVP Engineering and Product, Docker

Cloud Run, your home for AI applications

Support for the compose spec isn’t the only AI-friendly innovation you’ll find in Cloud Run. We recently announced general availability of Cloud Run GPUs, removing a significant barrier to entry for developers who want access to GPUs for AI workloads. With its pay-per-second billing, scale to zero, and rapid scaling (which takes approximately 19 seconds for a gemma3:4b model for time-to-first-token), Cloud Run is a great hosting solution for deploying and serving LLMs. 

This also makes Cloud Run a strong solution for Docker’s recently announced OSS MCP Gateway and Model Runner, making it easy for developers to take the AI applications locally to production in the cloud seamlessly. By supporting Docker’s recent addition of ‘models’ to the open Compose Spec, you can deploy these complex solutions to the cloud with a single command.  

Bringing it all together

Let’s review the compose file for the above demo. It consists of a multi-container application (defined in services) built from sources and leveraging a storage volume (defined in volumes). It also uses the new models attribute to define AI models and a Cloud Run-extension defining the runtime image to use:

Building the future of AI

We’re committed to offering developers maximum flexibility and choice by adopting open standards and supporting various agent frameworks. This collaboration on Cloud Run and Docker is another example of how we aim to simplify the process for developers to build and deploy intelligent applications. 

Compose Specification support is available for our trusted users — sign up here for the private preview.

AI is making it easier than ever to get clear, reliable answers from your data. With intelligent tools like the Conversational Analytics API, powered by Gemini, you no longer need intricate systems to get insights.

The Conversational Analytics API lets you use everyday language to ask questions of your data in BigQuery or Looker, with more sources to come. This brings powerful business intelligence to your entire organization through the apps you already use, like Slack. Developers can also build custom data agents that understand your business and provide answers where they’re needed. 

Now in preview, this API transforms unstructured conversation and structured data into actionable insights, accessible through natural language. You can create custom data agents, map business terms, and define calculations to empower your users and analysts.

Conversational Analytics API’s intelligent toolkit

The Conversational Analytics API integrates multiple AI-powered tools to process user requests, including Natural Language to Query (NL2Query) and a Python code interpreter for generating responses. A critical context retrieval tool is also utilized to ensure the API’s answers are accurate and relevant by incorporating details about your specific datasets. 

Let’s take a peek under the hood:

Context retrieval: The key to unlocking relevant answers

For intelligent conversational data interactions, leveraging the context retrieval tool is essential. For BigQuery, the tool meticulously pulls schema information, detailed column, table descriptions from Dataplex. When interacting with Looker, it accesses the LookML model, retrieving field definitions, labels, and defined measures. This deep, accurate understanding of your data’s structure, relationships, and inherent business logic is critical as it enables the agent to become an expert in your data landscape. This ensures every response is firmly grounded, highly relevant, and ultimately, a trusted answer for your business, providing reliable and trusted answers.

NL2Query engine: Turning questions into queries

At the core of the Conversational Analytics API lies a robust NL2Query engine, designed to support both BigQuery and Looker data sources. This engine translates user-provided natural language questions into semantically equivalent and syntactically correct queries appropriate for the specified data source. 

For instance, a user could pose a question such as, “What is the average sales value per order item broken down by payment method?” and the engine would process this to generate and execute the necessary query, providing a precise answer without requiring the user to write any SQL. The NL2Query engine’s capability includes handling ambiguity and inferring the customer’s intent from natural language input, ensuring accurate mapping to the underlying dataset structure.

Python code interpreter: Unleash advanced analytics

Building upon basic query capabilities, the code interpreter functionality within the Conversational Analytics API leverages Python to facilitate advanced analytical tasks. This allows the API to generate and execute Python code for complex calculations and statistical analyses, going beyond what standard query languages can achieve. Users can address intricate scenarios that would be difficult or impossible with SQL alone, accessing sophisticated analyses like statistical modeling and data transformations through Python libraries, all via a conversational interface without needing direct Python coding.

For instance, a user could request to “Calculate churn rate by customer segment, and show me the distribution.” In response, the code interpreter would automatically generate and execute the necessary Python code to perform this analysis and deliver insightful visualizations and statistics.

If you are Interested in testing out the code interpreter tool, currently in preview. Learn more here.

Intelligent visualization engine for data stories that pop

Data in its raw form can often be challenging to interpret. The visualization engine within the Conversational Analytics API addresses this by transforming the results of your queries into compelling visual representations. This capability streamlines the data visualization process, providing benefits such as instant chart generation without the manual steps of selecting data, labeling axes, or choosing chart types and colors. The result is the transformation of numerical data into visually engaging and easily digestible insights that effectively highlight hidden patterns and trends.

For example, a user could request to “Plot total orders by month,” and the system would instantly generate a relevant visualization, such as a line chart, effectively illustrating sales performance over time, ready for sharing and analysis.

Start having intelligent conversations with your data

Stop struggling with data complexity. Request a demo today or turn on Conversational Analytics in Looker to begin chatting with your data. You can even bring Conversational Analytics to your own application with this API. Get started in minutes with our quickstart application here.

When it comes to large-scale data analysis, BigQuery is a powerhouse, with fundamental aggregate functions like SUM, AVG, and COUNT allowing you to extract meaningful insights for all types of workloads. And today, we’re excited to take your data analytics to the next level with a suite of advanced aggregation features that unlock more complex and sophisticated use cases. These aggregate functions fall into three categories: 

• Group by extension (grouping sets/cube, group by struct, array, group by all)

• User-defined aggregate functions (Javascript/SQL UDAFs) 

• Approximate aggregate functions (KLL quantiles, Apache DataSketches)

We built these functions as they were top feature requests from our customer council group. Here is what New York Times had to say:

“I just want to say thank you to BigQuery’s team who launched GROUP BY ROLLUP. We had a daily query taking more than 2 hours to run that now takes 10 minutes using this instead, and many other teams that want to use it now because of it. We saw slot consumption drop by about 96%!” – Edward Podojil, consultant, The New York Times

Let’s take a deeper look at these new aggregate functions, and how to use them in your data analytics workflows. 

Group-by extensions

Grouping sets provide flexibility in calculating aggregations in multiple dimensions in one single statement without having to use UNION ALL. Group by struct, array allows you to easily group by commonly used data types in BigQuery. Group by lets you group by all non-aggregate columns in the Select statement without having to repeat each column twice. 

GROUP BY GROUPING SETS, CUBE (GA) 

Users often need to slice and dice their data in multiple dimensions. Currently you have to rely on repeated UNION ALL/CROSS Join to group your data, which can make queries cumbersome and hard to understand. GROUP BY GROUPING SETS allow you to group different dimensions in a single statement. For example, the following query give you the sum of amount grouped by different combinations: 

• Date: total sales per day

• Region: total sales per region

• Product: total sales per product

GROUP BY CUBE(x, y) is a shorthand syntax for GROUP BY GROUPING SETS ((x,y), x, y, ()), so you can GROUP BY all combinations of different dimensions.

GROUP BY STRUCT, ARRAY (GA)
STRUCT and ARRAY are among the most commonly used data types in BigQuery today. Working with STRUCT and ARRAY data in BigQuery just got easier! You can now use GROUP BY and SELECT DISTINCT for these semi-structured data types directly. This means no more time-consuming workarounds like converting STRUCT/ARRAY to JSON strings. This will simplify your queries and boost performance, making complex analysis more efficient (documentation).

GROUP BY ALL (GA)
GROUP BY ALL deduces non-aggregated columns from the SELECT clause, eliminating the need to list the same columns twice in SELECT and GROUP BY. It’s often used in queries with many dimensions and few aggregations. Listing all columns twice would be long and tedious. For example, you can group Chicago taxi trips by company, payment_type and taxi_id with a simple GROUP BY ALL query.

User-defined aggregate functions (Javascript/SQL UDAF)

User-defined aggregate functions, or UDAFs, let you define custom aggregations once, and reuse them across projects and teams. You no longer need to write repeated logic again and again to unlock advanced functionality like weighted average, merging JSON data, or even simulating geospatial functions.

Javascript UDAF (GA)

JavaScript user-defined aggregate functions (JS UDAFs) let you create custom aggregation logic beyond built-in functions, so you can calculate metrics tailored precisely to your needs. For instance, you can craft UDAFs to compute weighted averages, specialized statistics, or even construct data sketches. Here’s an example of a JavaScript UDAF that simulates the mode() function, returning the most frequent value within a group.

SQL UDAF (GA)

SQL user-defined aggregate functions (SQL UDAFs) allow users to encapsulate complex aggregate expressions into a UDF, for composability and reusability without having to write the same code again and again. For example, you can wrap multiple aggregate function calls into a UDAF through a struct constructor.

Approximate aggregate functions 

Companies across ad-tech, retail, fin-tech usually have massive amounts of multi-dimensional data. However, extracting data insights from millions or billions of rows of user-behavior data can be very expensive and time-consuming. Many companies are willing to calculate approximate aggregated results within defined error bounds, to get faster responses. Sketches enable approximate estimates of distinct counts, quantiles, histograms, and other statistical measures — all with minimal memory and computational overhead, and with a single pass through the data at scale. 

KLL quantile functions (Preview)

BigQuery supports quantile calculations using native KLL quantile functions. For example, you could estimate the median trip duration and trip distance for all taxi rides on a given day or month. 

First, create daily KLL quantile sketches over trip_seconds and trip_miles from the `chicago_taxi_trips` data table. 

Then, with EXTRACT_POINT_INT64/FLOAT64 functions, you can get the median trip_seconds and trip_miles for all trips on a specific day.

You can also leverage the MERGE_POINT_INT64/FLOAT64 functions to estimate the median trip_seconds and trip_miles for all trips in a month. The query merges daily sketches at first, and then calculates quantile values over a month. 

Apache DataSketches (GA) 
Besides native support for sketches, BigQuery also supports open-source Apache DataSketches, a high-performance library of stochastic streaming algorithms. Initially developed by Yahoo, you can directly use them though the public UDF bigquery-utils repo powered by JS UDAFs. Here are some examples, just name a few (see more details in this blog post): 

1. Theta Sketch: designed for cardinality estimation and set operations (union, intersection, difference)

2. KLL Sketch: designed for quantile estimation

3. Tuple Sketch: an extension of the Theta Sketch that supports associating values with the estimated unique items

The following examples use Theta Sketch over a public dataset, Chicago taxi trips, to demonstrate the power of the UDAF sketches library. 

First, create a daily Theta Sketch to estimate the unique number of taxis running per day, using theta_sketch_agg_string.

You can estimate the unique number of taxis that run on both days, using theta_sketch_intersection.

You can estimate the unique number of taxis that run on either of these two days, using theta_sketch_union.

Do more with advanced aggregation

Using BigQuery’s advanced aggregation capabilities allows you to perform more complex analysis efficiently. Grouping sets provide flexibility in calculating aggregations at multiple dimensions. UDAFs empower you to define custom aggregations. Approximate aggregate functions provide speed, scalability and performance when approximate results are acceptable. By leveraging these features, you can unlock deeper insights from your data and help your team make timely business decisions. We can’t wait to hear your use cases and how you plan to use them in your day to day analysis!

Accurate time-series forecasting is essential for many business scenarios such as planning, supply chain management, and resource allocation. BigQuery now embeds TimesFM, a state-of-the-art pre-trained model from Google Research, enabling powerful forecasting via the simple AI.FORECAST function.

Time-series analysis is used across a wide range of fields including retail, healthcare, finance, manufacturing, and the sciences. Through the use of forecasting algorithms, users can have a more thorough understanding of their data including the recognition of trends, seasonal variations, cyclical patterns, and stationarity. 

BigQuery already natively supports the well-known ARIMA_PLUS and ARIMA_PLUS_XREG models for time-series analysis. More recently, with the rapid progress and success of large pre-trained LLM models, the Google Research team developed TimesFM, a foundational model specifically for the time series domain.

The Time Series foundation model

TimesFM is a forecasting model that’s pre-trained on a large time-series corpus of 400 billion real-world time-points. A big advantage of this model is its ability to perform “zero-shot” forecasting. This means that it can make accurate predictions on unseen datasets without any training. In terms of the architecture, TimesFM is built as a decoder-only transformer model, which outputs batches of contiguous time-point segments at a time. This model has been featured on the GIFT-Eval benchmark and Monash public dataset, with a variety of public benchmarks from different domains and granularities. While ARIMA_PLUS offers customizability and explainability, the TimesFM model provides high ease-of-use and delivers good generalizability across many business domains, and often beats custom trained statistical and deep learning models.

How BigQuery supports TimesFM

The latest TimesFM 2.0 is now a native model in BigQuery. With 500 million parameters, TimesFM model inference runs directly on BigQuery infrastructure so there are no models to train, endpoints to manage, connections to set up, or quotas to adjust. TimesFM in BigQuery is also fast and scalable — you can forecast millions of univariate time series in a few minutes with a single SQL query.

Examples of the new AI.FORECAST function

To demonstrate, consider a use case that relies on the public bigquery-public-data.san_francisco_bikeshare.bikeshare_trips table. This dataset contains information about individual bicycle trips taken using the San Francisco Bay Area’s bike-share program. 

Example 1: Single time series

The following query aggregates the total number of bike-share trips on each day and forecasts the number of trips for the next 10 days (the default horizon).

The results look similar to:

The output includes the forecast timestamp and values through the columns forecast_timestamp and forecast_value. The confidence_level is default as 0.95. The prediction_interval_lower_bound and prediction_interval_upper_bound show the bounds for each forecasted point.

Example 2: Multiple time series

The AI.FORECAST function also lets you forecast multiple time series at a time as shown in the following example. The following query forecasts the number of bike share trips per subscriber type and per hour for the next month (approximately 720 hours), based on the previous four months of historical data.

The results look similar to the following:

In addition to the columns used by the single time series, there’s the time series identifier column, which we defined earlier as subscriber_type.Visualize the results 

You can merge the history data and forecasted data and visualize these results. The following graph visualizes the ‘Subscriber’ time series with its the lower and upper bounds of the prediction interval as follows:

You can see the detailed queries we used to generate this in the tutorial.

When to use TimesFM vs ARIMA_PLUS

For quick, out-of-the-box forecasts, establishing baselines, or identifying general trends with minimal setup, use TimesFM. If you need to model specific patterns, fine-tune forecasts for seasonality or holidays, multivariate (ARIMA_PLUS_XREG), require explainable results, or want to leverage a longer historical context, ARIMA_PLUS is the more suitable choice.

Take the next step

The TimesFM 2.0 model is now available in BigQuery in preview. For more details, please see the tutorial and the documentation.

Today, we’re announcing Sol, a new transatlantic subsea cable system to connect the U.S., Bermuda, the Azores, and Spain. Sol translates to “sun” in Spanish and Portuguese, a nod to the cable system’s landing points in warmer climates. Alongside the Nuvem subsea cable, Sol completes our unique investment in transatlantic resiliency — with the two systems interconnecting terrestrially in the U.S. and in Iberia, as well as in Bermuda and the Azores. 

The Sol cable will be manufactured in the U.S. and, once operational, bolster capacity and reliability for our growing network of 42 Google Cloud regions around the world, helping meet growing customer demand for Google Cloud and AI services across the U.S., Europe, and beyond.  

Palm Coast, Florida, will anchor the Sol cable in the U.S. We’ll partner with DC BLOX to land the cable and establish a new connectivity hub in the Sunshine State. We’ll also develop a terrestrial route linking Palm Coast to our South Carolina cloud region.

When it’s complete, Sol will be the only in-service fiber-optic cable between Florida and Europe. In Spain, we’ll partner with Telxius to provide the necessary infrastructure to land the Sol cable in Santander, further integrating the Google Cloud region in Madrid into our global network. 

“Florida is once again at the forefront of strengthening our nation’s digital infrastructure. The anchoring of the Sol subsea cable in our state is a testament to our state’s commitment to fostering an environment that is ripe for technological investment and innovation. We are proud to partner with companies like Google and DC BLOX on strategic infrastructure projects that will support the future of AI innovation and bring tangible benefits to our communities and the entire Sunshine State.” – J. Alex Kelly, Florida Secretary of Commerce

“This is a landmark moment for Palm Coast, Flagler Beach, and Flagler County, and it’s a clear signal that we are a community of the future, investing in our economic development and vitality. The Sol subsea cable is more than just infrastructure; it’s a gateway to unprecedented global connectivity that will attract further high-caliber industries that our residents deserve. We are not just putting Palm Coast and our community on the map; we are building a direct route to the world’s digital economy, ensuring a prosperous and dynamic future for our community.” – Vice Mayor Theresa Carli Pontieri, Palm Coast, Florida 

 “DC BLOX is proud and honored to expand the foundational digital infrastructure that is vital to Florida’s growing economy. Supporting Google’s Sol cable, along with the capacity for additional cables, the new Palm Coast Cable Landing Station campus enhances Florida’s position as a nexus for global communications.” – Chris Gatch, Chief Revenue Officer, DC BLOX 

“Digital connectivity is today a strategic factor for the economic and social development of any city. At Santander City Council, we are committed to facilitating this type of infrastructure because its implementation attracts investment, talent, and technology companies, in addition to generating employment and positioning us as a city prepared for remote work, advanced services, and the Smart City model. That’s why we appreciate the interest that Google has shown in our coast and for choosing Santander as a transatlantic connection point!” – City Council Mayor Gema Igual, Santander, Spain, 

“Telxius is delighted to support Google in bringing the Sol cable to Spain, a key milestone that will boost transatlantic connectivity with enhanced capacity, reliability, and resilience. Through our submarine landing services, we accelerate project implementation and help expand global networks for our customers.” – Mario Martín, CEO, Telxius

“The Government of the Azores welcomes this important investment in the region with great enthusiasm and congratulates Google on its visionary, pioneering, and strategic approach, which fully aligns with our own vision: that the Azores are an important and strategic hub for digital connectivity in Europe and the North Atlantic.” – Artur Lima, Vice President, Government of the Azores

“Promoting the expansion and modernization of submarine cable infrastructure, with a view to strengthening the country’s international connectivity and digital competitiveness, is an essential goal for the government to the development of our country. The project for Google’s new “Sol” cable, with 16 fiber optic cable pairs and the potential to provide redundancy to the “Nuvem” cable, also from that company, will increase resilience and respond to the growing demand for digital infrastructure in Portugal and Europe.”– Miguel Pinto Luz, Minister of Infrastructure and Housing, Portugal

“Bermuda welcomes Google’s continued investment with the announcement of a second cable ‘Sol,’ further establishing Bermuda as a digital hub in the Atlantic. Bermuda greatly values its positive relationship with Google. This second cable highlights the benefits of this Government’s strategy to develop Bermuda as a progressive world leading jurisdiction for technology companies.” – Alexa Lightbourne, Minister of Home Affairs, Bermuda

Sol will add capacity, increase reliability, and decrease latency for Google users and Google Cloud customers around the world. Alongside cable systems like Nuvem, Firmina, Equiano, and Grace Hopper, Sol further establishes key locations across the Atlantic as connectivity hubs, strengthening local economies and bringing AI’s benefits to people and businesses around the world.

There’s a buzz of excitement here at Tobacco Dock as we welcome our customers and partners to the Google Cloud Summit London. Together, we’re exploring the essential role Google Cloud is playing in driving AI innovation and adoption across the UK. Today is about shining a spotlight on our customers and partners, focusing on the real-world stories of how they are accelerating innovation, realising value, and reshaping entire industries with the powerful combination of AI agents and robust cloud infrastructure.

The opportunity ahead is immense. AI is poised to inject more than £400 billion into the UK economy by 2030, and this year’s summit reaffirms our long-term commitment to helping the UK realise its potential and shape its AI future.

Investing in Britain’s digital future

To lead in the AI era, a nation needs a strong digital foundation. That’s why we’re proud to announce a series of strategic commitments to the UK’s digital future.

First, we are launching a new partnership with the UK Government to help modernise public services. This collaboration marks a significant shift, exploring the use of advanced tech and AI to move beyond decades-old legacy contracts that can leave essential services vulnerable. It supports the UK Government’s “blueprint for a modern digital government” by aiming to streamline services, reduce costs, and lay the digital foundations for AI success.

Equally important, we’re investing in a world-class, AI-ready workforce. Today, we are announcing a landmark proposal to provide free cloud and AI skills training for up to 100,000 UK public sector workers. This initiative is designed to accelerate the digital transformation of public services and empower the workforce with the talent to lead in the AI era.

These commitments are backed by our continued investment in critical infrastructure. We are close to reaching a major milestone with our data center in Waltham Cross, which will be fully operational by the end of the year. Delivering on our promise, this facility will provide British businesses with the high-performance, low-latency cloud they need to compete globally.

From AI vision to real-world value

Agentic AI is emerging as a major force in enterprise technology, representing a foundational shift in how work gets done. We’re helping British businesses go beyond experimentation and embrace a new era of intelligent agents. With advanced models like Gemini on our Vertex AI platform, we’re giving businesses the toolkits to drive real results and realise value faster.

At the Summit, we’re showcasing how leading UK companies are already leveraging these capabilities to drive impact:

• Imperial War Museums is the first UK museum to use AI at scale to digitise and transcribe oral histories. Implemented by our partner Capgemini, the Gemini-engineered system will make more than 20,000 histories from 1945 to the early 2000s searchable and accessible.

• LUSH is taking advantage of Vertex AI and Cloud Storage to reduce packaging waste through AI-powered product recognition at checkout in its beauty shops.

• Morrisons’ new in-app Product Finder—built with BigQuery and Gemini—helps customers quickly locate products in its grocery stores. It was used 50,000 times a day during the most recent Easter period, proving its immense practical value.

• Starling Bank launched “Spending Intelligence,” an industry-leading AI tool powered by Gemini models. Customers can now understand their spending habits simply by asking natural language questions and getting instant analysis.

• Toolstation now offers customers highly accurate product search results online and in-store, powered by Vertex AI Search for Commerce, which is boosting customer satisfaction and sales.

Extending Our Impact Through a Trusted Ecosystem

Collaboration is fundamental to progress. The achievements we celebrate today wouldn’t be possible without our incredible network of partners. Our work with Capgemini is helping British-born companies like Additive Catchments, a majority non-profit-owned company, to deploy AI-powered water stewardship solutions. Leveraging Google Cloud, they provide real-time insights to improve water quality and safeguard ecosystems across the UK—a perfect example of technology for good. 

This kind of innovation isn’t happening in isolation. Since 2023, more than 60% of UK generative AI startups have chosen Google Cloud due to our deep commitment to the ecosystem. From our ‘Gemini for UK’ initiative, which offers up to £280,000 in cloud credits and access to expert mentorship and training, to our recent partnership with Tech London Advocates, our aim is to help founders scale faster. 

We’ve also secured spaces for top UK AI Startups on the Europe-wide Google for Startups accelerator. These companies, such as Martee ai, are on a mission to improve the face of grab-and-go food, for the benefit of people and the planet, and Dyad AI, which is helping healthcare providers improve care quality and automate administrative workflows. They will be joining 15 top AI startups from across Europe in receiving expert help from Google so they can take their startups to the next level.

Setting a Secure Foundation for a Bold Future

A big part of our mission is to be the most trusted cloud partner for businesses, exemplified through our leading security, data sovereignty, and responsible AI practices. At the summit, Lloyds Banking Group and Vodafone are sharing how they are using Google Cloud’s security tools to help protect their operations and innovate securely for millions of customers.

This deep commitment to security is why Google Cloud announced plans earlier this year to acquire Wiz. By integrating Wiz’s best-in-class, multi-cloud visibility with Mandiant’s unmatched threat intelligence, we will offer UK businesses proactive, comprehensive protection across their entire cloud estate.

For UK organizations, including the public sector, we’re also building trust in generative AI by ensuring data sovereignty and compliance, so they can use this technology while keeping their data completely secure. Today we are happy to share that Google Cloud is strengthening its UK data residency commitment, empowering organizations with the choice to keep Gemini 2.5 Flash processing entirely within the UK. This expansion opens up even more use cases for businesses operating here.

This combination of world-class security and a commitment to openness is the foundation upon which UK organizations can innovate with confidence. Our leadership in open-source technologies like Kubernetes enables them to build and run applications anywhere, knowing their data is protected by world-class security. Whether it’s a bank securing its services, a retailer reimagining the customer experience, or a startup building the next big thing, our goal is to provide the tools and the trust to build boldly. 

With that mission in mind, UK businesses have a real opportunity to lead on the global stage. Together, with our customers, partners, and the public sector, we’re building much more than a cloud platform; we’re helping UK organisations not only stay ahead but to move boldly into the future and shape the AI era.

Didn’t make it to London? Please browse our catalogue of upcoming events & sign up to join us at another exciting gathering of cloud and AI leaders from across industries and around the world.

Today, we’re making it even easier to achieve breakthrough performance for your AI/ML workloads: Google Cloud Managed Lustre is now GA, and available in four distinct performance tiers that deliver throughput ranging from 125 MB/s, 250 MB/s, 500 MB/s, to 1000 MB/s per TiB of capacity — with the ability to scale up to 8 PB of storage capacity. The Managed Lustre solution is powered by DDN’s EXAScaler, combining DDN’s decades of leadership in high-performance storage with Google Cloud’s expertise in cloud infrastructure.

Managed Lustre provides a POSIX-compliant, parallel file system that delivers consistently high throughput and low latency, essential for:

• High-throughput inference: For applications that require near-real-time inference on large datasets, Lustre provides high parallel throughput and sub-millisecond read latency.

• Large-scale model training: Accelerate the training cycles of deep learning models by providing rapid access to petabytes-sized datasets. Lustre’s parallel architecture ensures GPUs and TPUs are fed with data, minimizing idle time.

• Checkpointing and restarting large models: Save and restore the state of large models during training faster, improving goodput and allowing for more efficient experimentation.

• Data preprocessing and feature engineering: Process raw data, extract features, and prepare datasets for training, reducing the time spent on data pipelines.

• Scientific simulations and research: Beyond AI/ML, Lustre excels in traditional HPC scenarios like computational fluid dynamics, genomic sequencing, and climate modeling, where massive datasets and high-concurrency access are critical.

Lustre is designed for the highly parallel and random I/O that characterizes many AI/ML training and inference tasks. This parallel processing capability across multiple clients ensures your compute resources are never starved for data.

Performance tiers and pricing

Managed Lustre offers flexible pricing and performance tiers designed to meet the diverse needs of your workloads, whether you’re focused on capacity or highest throughput density. 

Please see more details at the Managed Lustre pricing page.

Irrespective of the aggregate throughput, all tiers come with sub-millisecond read latency, high single-stream throughput, and are perfect for parallel access to many small files.

Driving innovation together: partnering with DDN

Google Cloud’s Managed Lustre is powered by DDN’s EXAScaler, bringing together two industry leaders in high-performance computing and elastic cloud infrastructure. This partnership represents a joint commitment to simplifying the deployment and management of large-scale AI and HPC workloads in the cloud, thanks to:

• Trusted leaders: By combining DDN’s decades of expertise in high-performance Lustre with Google Cloud’s global infrastructure and AI ecosystem, we are delivering a foundational capability that removes storage bottlenecks and helps our customers solve their most complex challenges in AI and HPC.

• Fully managed and supported solution: Enjoy the benefits of a fully managed service from Google, with comprehensive support from both Google and DDN, for seamless operations and peace of mind.

• Global availability and ecosystem integration: Managed Lustre is now globally accessible in multiple Google Cloud regions and integrates with the broader Google Cloud ecosystem, including Google Kubernetes Engine (GKE) and TPUs.

These benefits caught the attention of one of our largest partners, NVIDIA, who is looking forward to having it as part of its NVIDIA AI platform. 

“Enterprises today demand AI infrastructure that combines accelerated computing with high-performance storage solutions to deliver uncompromising speed, seamless scalability and cost efficiency at scale. Google and DDN’s collaboration on Google Cloud Managed Lustre creates a better-together solution uniquely suited to meet these needs. By integrating DDN’s enterprise-grade data platforms and Google’s global cloud capabilities, organizations can readily access vast amounts of data and unlock the full potential of AI with the NVIDIA AI platform (or NVIDIA accelerated computing platform) on Google Cloud — reducing time-to-insight, maximizing GPU utilization, and lowering total cost of ownership.” – Dave Salvator, Director of Accelerated Computing Products, NVIDIA

Get started today!

Ready to supercharge your AI/ML and HPC workloads? Getting started with Managed Lustre is simple:

1. Navigate to Managed Lustre in the Google Cloud console.

2. Provision your Managed Lustre instance, choosing the performance tier and size that best fits your needs.

3. Connect your compute instances, GKE clusters to your new high-performance file system.

For detailed instructions and documentation, please visit the Managed Lustre documentation. And if needed, reach out to Google Cloud sales specialists.

Watch the Fireside Chat

Don’t miss the opportunity to learn more about the strategic partnership between Google Cloud and DDN, and the unique capabilities of Managed Lustre. Read the official DDN press release here.

Watch the fireside chat with Sameet Agarwal, VP/GM Storage and Sven Oehme, CTO of DDN, here.

Today, we are thrilled to announce the expansion of the Z3 Storage Optimized VM family with the general availability of nine new Z3 virtual machines that offer local SSD capacity ranging from 3 TiB to 18 TiB per VM, complementing existing Z3 VMs which offer 36TiB of Local SSD per VM. We are also very pleased to launch a Z3 bare metal instance, which includes up to 72 TiB of Local SSDs. Z3 VMs enable customers like Shopify, Tenderly and ScyllaDB to achieve impressive performance improvements for their high performance storage workloads by reducing the IO access latency by up to 35% compared to VM instances using previous-generation local SSDs. 

Z3 VMs are designed to run I/O-intensive workloads that require large local storage capacity and high storage performance, including SQL, NoSQL, and vector databases, data analytics, semantic data search and retrieval, and distributed file systems. The Z3 bare metal instance provides direct access to the physical server CPUs and is ideal for workloads that require low-level system access like private and hybrid cloud platforms, custom hypervisors, container platforms, or applications with specialized performance or licensing needs. 

Both Z3 VMs and the bare metal instance are based on Titanium SSDs, which offload local storage processing from CPU resources to deliver real-time data processing, low-latency, high-throughput storage performance and enhanced storage security. Z3 VMs with Titanium SSD offer up to 36 GiB/s of read throughput and up to 9M IOPS, increasing write storage performance by up to 25% compared to previous generation Local SSDs¹.

Based on the 4th Gen Intel Xeon scalable processor, Z3 VMs come with up to 176 vCPUs, 1,408 GiB of memory, and 36 TiB of local storage in 11 virtual machine shapes. The Z3 bare metal instance offers 192 vCPUs, 1,536 GiB of memory and 72 TiB of local storage. Z3 VMs and the bare metal instance deliver the connectivity and storage performance that enterprise workloads need, with up to 100 Gbps in standard bandwidth and up to 200 Gbps with Tier1 networking for high-traffic applications.

The expanded Z3 virtual machine portfolio lets you rightsize your infrastructure and scale your clusters to meet workloads requirements by providing larger total local SSD capacity and higher local SSD capacity per vCPU. Z3 offers two different VM types: the standardlssd VM types, which include five VM shapes that offer about 200 GiB of local SSD per vCPU. They are optimized for data analytics (OLAP), and SQL databases like MySQL and Postgres workloads. The highlssd VM types include six different VM shapes and the Z3 bare metal instance. They offer about 400 GiB of local SSD per vCPU and are optimized for distributed databases, data streaming, large parallel file systems and data search.

What our customers and partners are saying

“We are thrilled to announce Nutanix Cloud Clusters coming to Google Cloud at the end of CY25 as part of Nutanix’s commitment to delivering flexible, hybrid cloud solutions. Google Cloud’s Z3 instance types represent a perfect foundation for Nutanix to enable performance and resilience for enterprise applications. We’re excited about our partnership with Google Cloud in empowering our joint customers with greater choice and simplicity in their cloud journey.” – Saveen Pakala, Vice President of Product Management, Nutanix

“OP Labs contributes to the Optimism protocol, which enables orders of magnitude of improved performance and scalability for Ethereum. Z3 reduces p99 block insertion tail latencies by 30-50% for our most I/O-demanding blockchain nodes compared to N2. By migrating our solution to Z3, we will be able to scale our blockchain nodes to handle L2 state growth in a more performant and cost-effective way.” – Zach Howard Senior Staff Engineer, OP Labs

The launch of Google Cloud’s Z3 storage optimized instances with smaller VM shapes  represents a leap forward in performance for high-traffic NoSQL environments. In internal tests and customer projects, ScyllaDB has impressively leveraged the advantages of Z3 including extremely low latencies under high read and write loads, high IOPS capacity enabling the processing of massive amounts of data and excellent cost-performance ratio for large-scale production systems. We are very excited to offer Z3 family servers in ScyllaDB Cloud, including Bring Your Own Account (BYOA).” – Avi Kivity, Co-founder and CTO, ScyllaDB

“Shopify has found Z3s to be an excellent platform to build our most performance sensitive storage systems on. We experienced a critical need for both large data volumes while remaining sensitive to latency and throughput on the storage side. While Google has a lot of options, local SSD was really the best fit, and Z3s allowed us to achieve the best price/performance along with enhanced stability appropriate for a source of truth Storage workload. Right now we see these storage optimized VMs as our platform of choice for the future.” – Mattie Toia, VP Infrastructure, Shopify

“Tenderly is built to be your go-to for Web3 production and development, bringing all the necessary infrastructure into one place. This allows teams to operate with speed and confidence, making blockchain technology accessible. We’ve seen impressive results running blockchain workloads on Z3 instances, with a 40% improvement on read latency compared to N2 and N2D instances.” – Ilija Petrovic, SRE Lead, Tenderly

“The VAST AI Operating System gives organizations a unified platform to reason over all of their data – structured, unstructured, and streaming through a global namespace that spans cloud and on-prem environments – enabling intelligent agents and applications to operate with full context and real-time speed. ,For customers running on Google Cloud, Z3 VMs complement this vision by providing the ideal storage infrastructure to accelerate these workloads, ensuring AI pipelines run fast and scale effortlessly in the cloud.” – Renen Hallak, Founder & CEO, VAST Data

Z3 VMs are also the physical foundation of AlloyDB, our flagship PostgreSQL-compatible database service, delivering sophisticated multi-level caching. AlloyDB uses Z3’s expansive local SSDs as an ultra-fast cache, holding datasets up to 25x larger than can be stored in memory. Database queries can access these large, cached datasets at latencies that closely approach in-memory performance, particularly when factoring in overall end-to-end application response times. This is a significant advantage for very large databases, including real-time analytical workloads, as AlloyDB’s high-performance columnar engine operates entirely within this massive cache. AlloyDB on Z3 VMs will soon be available in preview, delivering up to 3x better performance than N-series VMs for transactional workloads, particularly for large datasets.

Enhanced maintenance experience

Z3 instances make it easier for you to plan ahead and schedule maintenance operations at a time of your choosing by providing notice from the system several days in advance of a required maintenance. The new Z3 VMs further enhance the maintenance experience by allowing you to live-migrate an instance during maintenance events for VMs with 18 TiB or less of local SSD storage. For Z3 VMs with 36 TiB of local SSD and for Z3 bare metal instances, you’ll also receive in-place upgrades that preserve your data through the planned maintenance events.

Support for Hyperdisk

Z3 VMs support Hyperdisk, Google Cloud’s workload-optimized block storage that lets you optimize the performance for each workload by independently tuning the storage performance and capacity for each instance.

Z3 VMs are compatible with Hyperdisk Balanced, Hyperdisk Throughput, and Extreme Hyperdisk storage for scalable, high-performance network-attached storage, supporting up to 512 TiB of capacity per instance. For general-purpose workloads, Hyperdisk Balanced, with up to 160K IOPS per instance, offers a mix of performance and cost-efficiency. Hyperdisk Extreme delivers ultra-low latency and supports up to 350K IOPS and 5,000 MiB/s throughput per Z3 VM instance and up 500K IOPS and 10,000 MiB/s throughput for the Z3 bare metal instance — making it well-suited for demanding workloads like databases. Using Hyperdisk for persistent storage and Z3 Local SSD for caching creates an optimal storage architecture for high end databases and mission critical workloads

Get started with Z3 today

Z3 VMs and bare metal instances are available today in most regions worldwide. To start using Z3 instances, select Z3 under the new Storage-Optimized machine family when creating a new VM or GKE node pool in the Google Cloud console. Learn more at the Z3 machine series page. Contact your Google Cloud sales representative for more information on regional availability.

^{1. Results are based on Google Cloud’s internal benchmarking}

Developers are racing to productize agents, but a common limitation is the absence of memory. Without memory, agents treat each interaction as the first, asking repetitive questions and failing to recall user preferences. This lack of contextual awareness makes it difficult for an agent to personalize their assistance–and leaves developers frustrated. 

How we normally mitigate memory problems: So far, a common approach to this problem has been to leverage the LLM’s context window. However, directly inserting entire session dialogues into an LLM’s context window is both expensive and computationally inefficient, leading to higher inference costs and slower response times. Also, as the amount of information fed into an LLM grows, especially with irrelevant or misleading details, the quality of the model’s output significantly declines, leading to issues like “lost in the middle” and “context rot”. 

How we can solve it now: Today, we’re excited to announce the public preview of Memory Bank, the newest managed service of the Vertex AI Agent Engine, to help you build highly personalized conversational agents to facilitate more natural, contextual, and continuous engagements. Memory Bank helps us address memory problems in four ways: 

• Personalize interactions: Go beyond generic scripts. Remember user preferences, key events, and past choices to tailor every response.

• Maintain continuity: Pick up conversations seamlessly where they left off across multiple sessions, even if days or weeks have passed.

• Provide better context: Arm your agent with the necessary background on a user, leading to more relevant, insightful, and helpful responses.

• Improve user experience: Eliminate the frustration of repeating information and create more natural, efficient, and engaging conversations.

Where you can access it: Memory Bank is integrated with the Agent Development Kit (ADK) and Agent Engine Sessions. You can define an agent using ADK, enable Agent Engine Sessions to store and manage conversation history within individual sessions. Now, you can enable Memory Bank to provide long-term memory for agents to store, retrieve, and manage relevant information across multiple sessions. You can also use Memory Bank to manage your memories with other agent frameworks including LangGraph and CrewAI.

Here’s how Memory Bank works

1. It understands and extracts memories from interactions: Using Gemini models, Memory Bank can analyze a user’s conversation history with the agent (stored in Agent Engine Sessions) to extract key facts, preferences, and context to generate new memories. This happens asynchronously in the background, without you needing to build complex extraction pipelines.

2. It stores and updates memories intelligently: Key information—like “My preferred temperature is 71 degrees,” or “I prefer aisle seats on flights” — is stored persistently and organized by your defined scope, such as user ID. When new information arises, Memory Bank (using Gemini) can consolidate it with existing memories, resolving contradictions and keeping the memories up to date.

3. It recalls relevant information: When a user starts a new conversation (session), the agent can retrieve these stored memories. This can be a simple retrieval of all facts or a more advanced similarity search (using embeddings) to find the memories most relevant to the current topic, ensuring the agent is always equipped with the right context.

This entire process is grounded in Google Research’s novel research method (accepted by ACL 2025), which enables an intelligent, topic-based approach to how agents learn and recall information, setting a new standard for agent memory performance.

Let’s take an example. Imagine you’re a retailer in the beauty industry. You have a personal beauty companion equipped with memory that recommends products and skincare routines.

As shown in the illustration, the agent is able to remember the user’s skin type (maintaining context) even after it evolves over time and be able to make personalized recommendations. This is the power of an agent with long-term memory.

Get started today with Memory Bank

You can integrate Memory Bank into your agent in two primary ways:

1. Develop an agent with Google Agent Development Kit (ADK) for an out-of-the-box experience

2. Develop an agent that orchestrates API calls to Memory Bank if you are building your agent with any other framework.

To get started, please refer to the official user guide and the developer blog. For hands-on examples, the Google Cloud Generative AI repository on GitHub offers a variety of sample notebooks, including integration with ADK and deployment to the Agent Engine runtime. For those wishing to try Memory Bank with third-party frameworks, we also provide notebook samples for LangGraph and CrewAI.

• Memory Bank with ADK agent

• Memory Bank with CrewAI agent

• Memory Bank with LangGraph agent

If you’re a developer using Agent Development Kit (ADK) but have never used Google Cloud before, you can still start by using our new express mode registration for Agent Engine Sessions and Memory Bank. Here’s how it works:

1. Sign up with your Gmail account to receive an API key

2. Use the key to access Agent Engine Sessions and Memory Bank

3. Build and test your agent within the free tier usage quotas

4. Seamlessly upgrade to a full Google Cloud project when you are ready for production

If you want to know more about Memory Bank, join the Vertex AI Google Cloud community to share your experiences, ask questions, and collaborate on new projects.

We’re thrilled to announce that Google has been named a Leader in the Gartner® Magic Quadrant™ for Search and Product Discovery.

We believe this recognition affirms Google’s evolving commitment to delivering AI solutions tuned to unique industry challenges, empowering businesses to transform the digital commerce experience and deliver high ROI-generating product discovery, through the use of AI in relevance, ranking, and personalization.

Built upon Google’s deep expertise & knowledge of the way users search, interact with & purchase products across a broad landscape of commerce domains, Vertex AI Search for commerce is a fully-managed, AI-first solution tailored for e-commerce. 

Redefining product discovery with generative AI

As digital channels continue to increase in traffic and adoption, businesses in e-commerce need to navigate the challenges involved in providing great digital experiences. Customers expect relevance, convenience, and personalization.

Vertex AI Search for commerce uses the best of Google AI to drive personalized product discovery at scale, optimized for revenue-per visitor. This enables businesses to not only understand user intent, but proactively surface the right products and content to shoppers through a variety of multimodal capabilities that leverage the latest advancements in generative AI. 

• Search and browse: Powering digital commerce sites and applications with Google-quality capabilities highly tuned for e-commerce and revenue maximization.

• Conversational search: Enabling real-time, back and forth conversation, powered by Gemini models to guide users through their shopping journeys, while optimizing for conversion.

• Recommendations: Deliver highly personalized recommendations at scale.

• Semantic image search: Users can search by image (“find this blouse”) or by a combination of text & image (“find the shoes that would look great with this blouse”).

In particular, our customers have noted our leadership in conversational search through one of our latest offerings, Conversational Commerce, which helps retailers provide assisted shopping on any digital channel, to engage with customers in a more natural and human-like conversation. This includes helping customers find their desired products online or helping a store associate answer questions using data from multiple sources to increase buying confidence for customers worldwide.

Intuitive search at the speed of intent, tuned for ecommerce

Search is changing as consumers are leveraging longer queries, seeking more comprehensive answers to questions.

Over the past decade, we’ve made significant strides in understanding user intent and refining our proprietary algorithms, now enhanced with gen AI. This has become increasingly important as user search behavior and the shopping journey have evolved, transforming the retail landscape. Unlike traditional models that are not AI-native, Vertex AI Search for commerce nearly eliminates the need for manual overriding to re-rank results and compensate for suboptimal search quality. Our solution is  developed to help businesses, from digitally native e-commerce companies to traditional retailers, adapt to this new era of buying and selling by leveraging AI to maximize revenue.

• Access to cutting edge AI/ML models: Vertex AI Search for commerce redefines e-commerce search using powerful AI / ML models like Gemini for unparalleled accuracy and relevance.

• Leverage Google’s proprietary intelligence: Vertex AI Search for commerce utilizes Google Shopping’s vast query & click datasets, along with our most advanced knowledge graphs, to train our industry-leading AI Relevance models.

• Advanced intent detection: Interprets complex queries and nuances in user intent beyond simple keyword matching, focusing on semantic meaning for intuitive results.

• AI-based catalog optimization:  Retail catalog search results are enhanced by combining Google Shopping’s effective web crawlers with Google’s extensive understanding of web content and our unique AI models.

• Deep and scalable personalization: Analyzes individual shopper behavior, preferences, and historical data to deliver tailored product recommendations and search rankings, boosting satisfaction, conversions, and loyalty.

• Managed deployment: Offers a fully managed solution for seamless integration and launch, minimizing engineering overhead.

• Flexible customization and control: Businesses can customize the search experience with specific business rules and optimization objectives, aligning with unique KPI goals and other unique metrics. 

The future of e-commerce is personalized and simplified

We believe being positioned as a Leader in the Gartner® Magic Quadrant™ for Search and Product Discovery underscores Google’s proven ability to deliver real business value. Vertex AI Search for commerce provides a comprehensive, AI-first solution that guides your customers through the buying journey, ensuring they find exactly what they need, every time. 

To download the full 2025 Gartner Magic Quadrant for Search and Product Discovery report, click here, and for more information on Vertex AI Search for commerce, see here or register for our upcoming product roadmap webinar. 

^{Gartner, Magic Quadrant for Search and Product Discovery – Mike Lowndes, Noam Dorros, Sandy Shen, Aditya Vasudevan, June 24, 2025}

^{Disclaimer: Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Google.}

^{GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, and MAGIC QUADRANT is a registered trademark of Gartner, Inc. and/or its affiliates and are used herein with permission. All rights reserved.}