Google Cloud

Though its name may suggest otherwise, Seattle Children’s is the largest pediatric healthcare system in the world. 

While its main campus is in its namesake city, Seattle Children’s also encompasses 47 satellite hospitals across Alaska, Montana, Idaho, and Washington, and patients come from as far away as Hawaii for treatment. For more than 100 years, Seattle Children’s has helped kids across the Western U.S. get healthy and stay healthy, regardless of the ability to pay.

With so much ground to cover and diverse patient populations to treat, Seattle Children’s has always looked to new technologies to bring improved, consistent care to its patients and providers. Generative AI is now the latest advance in their medical toolkit.

It started roughly two decades ago, when Seattle Children’s created its pediatric clinical pathways, a set of standardized protocols designed to help clinicians make quicker and more reliable decisions to address dozens of medical conditions. Such pathways were becoming commonplace across medicine, and Seattle Children’s had developed some of the first for children’s unique medical needs.

Innovative as these were, they still required clinicians to thumb through indexes and long binders of information to find what they needed for a given ailment. And in healthcare, it’s often the case that every second counts. 

Seattle Children’s was already working with Google Cloud on a number of projects, and as we began to explore the potential for generative AI to make the work of our clinicians easier, the clinical pathways seemed like an obvious place to start. Using  Vertex AI and Gemini, we were able to quickly develop our Pathways Assistant, which took training from the clinical pathways documentation and supercharged it with not just searchability but conversationality. 

Instead of flipping pages, we’d flipped the script on how quickly and reliably clinicians could find the lifesaving information they needed.

The pathways to improved healthcare run through Gemini
“Clinical pathways” are end-to-end treatment protocols for a specific condition or illness. Seattle Children’s pediatric clinical pathways are widely respected and used by hospitals around the globe, providing information on everything from diagnostic criteria to testing protocols to medication recommendations. 

Previously, these clinical pathways were documented exclusively in PDFs — hundreds of thousands of pages of them. Performing a traditional search of their contents for the answers clinicians needed delayed their ability to provide treatment in an environment where minutes or even seconds can be critical.

Google Cloud engineers worked with Seattle Children’s informatics physicians, who straddle the worlds of healthcare and technology, to create Pathway Assistant. The new multimodal AI chatbot that responds to spoken or written natural-language queries using the information in those PDFs.

After processing a question, Pathway Assistant searches each PDF’s metadata, which contains semi-structured data in JSON format that’s been extracted from the PDFs by Gemini and curated by clinicians. It then selects the most relevant PDFs, parses the information — including any complex flowcharts, diagrams, and illustrations embedded in them — and answers the clinician’s question in just a few seconds. 

Interactive information-finding for accurate decision-making
Pathway Assistant becomes more accurate with use. Healthcare providers can “discuss” clinical pathways with the chatbot, which, instead of answering a question, poses questions of its own if it needs clarification, going back and forth until it’s confident it can answer accurately.  The chatbot always displays the specific sections of each PDF that was the source for formulating  its answers, helping clinicians confirm the veracity of responses.

The interface also includes a way for users to provide feedback about the accuracy and appropriateness of the chatbot’s analysis and answers. The feedback is then logged in a BigQuery table for future forensic analysis — both by clinicians, who can query the information using natural language, and by the built-in Gemini models, which processes the feedback and summarizes what clinicians found confusing or how to improve the accuracy of future answers.

This reflexive capability enables Pathway Assistant to update the PDFs based on clinicians’ feedback if the inaccuracy stemmed from the PDF’s content. Clinicians are also finding that the metadata is becoming more accurate and requiring less curation. Pathway Assistant even corrects typos in the documentation automatically. And as new clinical pathways are developed, PDFs containing the latest information are added to the PDF library. 

This growing collection is housed securely in Google Cloud Storage, and the bigger it gets, the more useful it becomes — which wasn’t always the case. Whereas an expanding paper-based collection contained more information, it was also more material to wade through, which is especially challenging in emergency medical situations. Pathway Assistant almost entirely relieves this burden, synthesizing and delivering the most complete information at any time in a matter of seconds.

Ultimately, Pathway Assistant is not a decision-making tool but rather an information-finding tool. Research into critical, evidence-based guidelines that used to take hours now takes minutes. 

This speed and effectiveness helps clinicians make the right decisions more quickly at the point of care, drastically reducing research time and improving patient safety and outcomes. Ultimately, clinicians can  spend more time with more patients, not with more PDFs. 

Ask any physician, they’ll tell you that’s what the best medical technology enables them to do — focus on the patient, not paperwork.

In today’s world where instant responses and seamless experiences are the norm, industries like mortgage servicing face tough challenges. When navigating a maze of regulations, piles of financial documents, and the high stakes of homeownership, consumers quickly find that even simple questions can turn into complicated issues. And the same can be true for the customer reps trying to help them navigate all that complexity.

Like many enterprises, Mr. Cooper is exploring how agentic AI and advanced AI agents can help both our customers and employees meet their needs with confidence. In our work to develop just such an agent with Google Cloud, one of our curious discoveries has been that like a good team, the best AI agents may just be made up of groups of agents with distinct skillsets and abilities, and we come to the best results when they’re working in concert.

At Mr. Cooper, our mission is to “Keep the dream of homeownership alive.”We’re here to simplify the journey, provide clarity, and ensure our customers feel confident every step of the way. That confidence is key when they’re making one of the most consequential purchases, and decisions, of their lives. 

With those dual  goals of simplicity and certainty in mind, we partnered with Google Cloud to develop an agentic AI framework designed to complement and support our team. We call it the Coaching Intelligent Education & Resource Agent, or CIERA. We asked ourselves how to implement a chatbot that could effectively collaborate with our human agents to streamline both sides of the customer service experience.

And just as we prioritize hiring great groups of customer reps and mortgage agents, we’ve discovered how important it is to put together the right group of agents to effectively meet the needs of all our users. CIERA is designed to do exactly that, handling routine and time-consuming tasks to enhance efficiency, while empowering our people to focus on delivering what they do best — empathy, judgment and meaningful human connection. 

CIERA represent an exciting step forward in blending human expertise with AI capabilities, creating a collaborative approach that elevates both the customer experience and our team’s impact. And just as important as this work is for Mr. Cooper, CIERA also demonstrates how our multi-agent approach can serve as a model for companies across industries. Read on to learn how we did it, and how you can, too. 

The challenge: Beyond the reach of traditional automation 

Mortgage servicing is uniquely complex, where a customer might have a single question that requires an agent to cross-reference multiple documents. 

This presents several challenges for traditional automation:

• Siloed Knowledge: Scattered information makes it hard to see the full picture, but AI surfaces key data, helping agents make faster, smarter decisions for customers.

• Lack of Understanding: Traditional systems rely on rigid keywords and decision trees, often missing the true intent behind customer inquiries. Our AI framework uncovers context and intent, equipping agents with the insights they need to respond with empathy and accuracy.

• Inflexible Processes: When conversations take unexpected turns, legacy automation often fails, creating dead ends for customers and the team. AI provides real-time adaptive guidance, helping agents navigate these twists seamlessly.

To truly elevate the customer experience, we needed a solution capable of reasoning, orchestrating, and understanding context — one that enhances and amplifies our capabilities to deliver exceptional service. 

The vision: Introducing CIERA, a collaborative AI agent workforce

Our vision was to create an agentic framework that supports our call center agents by leveraging Google Cloud’s Vertex AI platform. CIERA’s AI agents handle repetitive and complex tasks, allowing our team to focus on what technology can’t. 

Guided by the principle that AI enhances human performance, these digital collaborators are designed to deliver accurate, comprehensive, and human-centered solutions.

Building the agent workforce: Our architectural blueprint
Our modular architecture assigns distinct roles to each AI agent, creating a scalable, efficient. and manageable system that seamlessly collaborates with people to make work smoother and more rewarding.

Meet the key players of our digital team and the solutions they deliver for team members and customers:

• Sage, the Head Agent: Sage monitors how all other AI agents perform. By learning from patterns across workflows, Sage helps ensure that each AI agent works in harmony with human teams. Key abilities include intelligent agent monitoring, recognizing useful trends and fine-tuning orchestration.

• Ava, the Orchestrator: Ava serves as the team’s coordinator, managing complex customer inquiries by breaking them into manageable tasks and assigning them to the appropriate AI assistants. While Ava doesn’t interact directly with customers, it ensures processes run smoothly, empowering human agents to remain central to delivering solutions.

• Lex, the Task Specialist: Lex specializes in complex tasks, helping human agents during customer calls by quickly offering insights to questions around loan applications or escrow analyses. Working behind the scenes, Lex provides insights that allow people to focus on connecting with customers and making informed decisions.
• Sky, the Data Specialist: Sky helps human teams navigate internal knowledge bases and FAQs. For questions about policies, procedures, or definitions, Sky provides accurate and timely information, freeing people to spend more time on meaningful interactions, rather than searching for data.

• Remy, the Memory Agent: Remy assists by remembering past actions and outcomes, which helps personalize workflows and inform future decisions. Remy’s memory supports ongoing learning and training, making it easier for human agents to access shared knowledge and continuously improve their skills.

• Iris, the Evaluation Agent: By evaluating confidence scores, detecting hallucinations, and grounding responses with Model Armor, Iris ensures consistency and authenticity, helping human agents provide trustworthy customer support.

How it works in practice: A real-life scenario

Imagine a customer initiates a call asking, “I received a notice my escrow payment is increasing. Can you explain why and tell me what my new total payment will be?”

Instead of relying solely on automated responses, CIERA ensures every step is grounded in close partnership between AI agents and human team members:

1. Orchestration: Ava receives the query, understands the two distinct parts (the “why” and the “what”), and creates a plan. Ava consults with a human agent, confirms the correct context and then delegates tasks to the Lex agents.
2. Parallel Processing: With human oversight, Ava assigns the “why” task to Lex, pointing it to the customer’s most recent escrow analysis document. Simultaneously, it tasks another Lex agent to calculate the new total payment based on data from our systems.
3. Synthesis: The Lex agent reads the document and reports back to the human agent: “The increase is due to a $200 annual rise in property taxes.” The other agent confirms the new total payment. The human similarly reviews the payment calculation before moving ahead.
4. Resolution: Ava gathers all AI-generated insights, but the human agent validates and personalizes the final response as needed to ensure clarity, empathy, and accuracy before delivering it to the customer.

This human-in-the-loop approach ensures complex, multifaceted questions are resolved with both the efficiency of advanced AI and understanding nuances with the trust that only people can provide. The partnership guarantees every answer is not just quick, but also trustworthy and tailored to the customer’s needs.

Ensuring quality and trust: The “agentic pulse” and human oversight

In a regulated industry like ours, trust and accuracy are non-negotiable. Deploying advanced AI requires an equally advanced framework for evaluation and governance. To achieve this, we developed two key concepts:

• The “Agentic Pulse” Dashboard: Our central command center for monitoring the health and performance of our agent workforce. Powered by model-based evaluation services within the Vertex AI platform , it goes beyond simple metrics. We track:
  • Faithfulness: Is the agent’s answer grounded in the source documents
  • Relevance: Does the answer directly address the customer’s question
  • Safety: Does the agent avoid generating harmful or inappropriate content
  • Business Metrics: How do we correlate these quality scores with classic KPIs like average handle time (AHT) and customer satisfaction (CSAT)
• The “Sandbox” for HITL: Our “Sandbox” environment provides space for our business and technical teams to safely review, test and refine agent processes. Additionally, if the “Agentic Pulse” flags an interaction for review, a human expert can analyze the agent’s reasoning and provide feedback, ensuring a continuous cycle of improvement and learning.

This robust governance framework gives us the confidence to deploy these powerful tools responsibly.

Projected impact: From complex processes to clear wins

While CIERA is on its journey towards full production, our projections based on extensive testing and modelling point to historic and transformative gains across the board:

• For our customers: We project a reduction in wait times and a higher rate of first-contact resolution, so customers can get answers quicker and with the benefits of round-the-clock support for many complex scenarios. 

• For our human agents: By automating tedious research, CIERA will free up our human agents to focus on sensitive and complex customer relationships that require a human touch and create better tools and resources for more engaging work.

• For our business: We anticipate a major reduction in average handling times for a large segment of inquiries and faster, more accurate resolutions that are a direct driver of customer happiness and loyalty.

Beyond mortgages: A blueprint for any complex industry

The architectural patterns developed with CIERA are not limited to mortgage servicing. This agentic approach — of using an orchestrator to manage a team of specialized AI agents—is a powerful blueprint that can be applied to any industry, including healthcare, logistics and manufacturing, by grappling with information and task complexity.

The future is agentic and collaborative

Our journey with CIERA is just beginning, but it has already solidified our belief that the future of customer service is agentic driven. By combining Mr. Cooper’s deep industry expertise with Google Cloud’s world-class AI infrastructure, we are not just building bots, we are cultivating a digital workforce.

This collaboration is about more than just lowering costs or improving efficiency — it’s about building trust, delivering clarity, and creating a customer experience truly worthy of the dream of homeownership.

^{The team would like to thank Googlers Sumit Agrawal and Crispin Velez and the GSD AI Incubation team for their support and technical leadership on agents and agent frameworks as well as their deep expertise in ADK, MCP, and large language model evaluations.}

Organizations today face immense pressure to secure their digital assets against increasingly sophisticated threats — without overwhelming their teams or budgets. 

Using managed security service providers (MSSPs) to implement and optimize new technology, and handle security operations, is a strategic delegation that can make internal security operations staff more efficient and effective.

At Google Cloud, we understand the value that MSSPs can bring, so we’ve built a robust ecosystem of MSSP partners, specifically empowered to help you modernize security operations and achieve better security outcomes, faster. 

MSSPs can help ease the pressure from three key challenges that can prevent organizations from staying ahead of threats, and achieving the cybersecurity outcomes they need to build operational resilience and power innovation to create growth.

Prolonged time to value and disruptive deployment: CIOs need security investments to demonstrate value quickly. Deploying new security solutions or migrating from existing ones can be costly and time-consuming, delaying the realization of benefits and increasing risk during transition periods. These complexities can lead to protracted implementation cycles, thereby delaying the realization of anticipated benefits and consequently increasing an organization’s risk exposure throughout the transition period. 

Limited resources, talent, and expertise: CISOs often find their teams stretched thin, struggling with the sheer volume of security alerts and manual tasks while often lacking specialized knowledge in emerging threat areas or modern security solutions. The demand for cybersecurity professionals continues to grow faster than the supply of qualified workers. The 2024 ISC2 Cybersecurity Workforce Study estimated a global workforce gap of 4.8 million professionals. 

High costs: CEOs often see the cost of building and maintaining internal resources to protect the business, and wonder why they’re not getting the expected return on their investment in terms of successful security outcomes. A purely in-house cybersecurity strategy demands substantial upfront capital investment, ongoing operational costs, and a significant commitment to recruiting and retaining highly specialized talent in a competitive market.

An expert, experienced member of the team armed with modern technology

Addressing these challenges effectively requires strategic investments and a clear understanding of where to allocate resources to achieve optimal security outcomes. Approved Google Security Operations MSSP partners are uniquely positioned to help you overcome these hurdles, combining their deep expertise with the power of Google Cloud Security products.

Accelerate time to value and deployment: Google Security Operations MSSP partners can help you to accelerate your security operations modernization journey with tailored migrations and efficient technology deployments. With thousands of security operations transformations and deployments under their belt, MSSP partners can get your company’s rules, detections, alerts, and telemetry sources in production quickly. 

Augment resources, talent, and expertise: By using best-in-class tooling like intelligence-driven and AI-enabled Google Cloud Security products, partners can filter out noise and escalate only issues requiring business context, helping to reduce the manual work your team faces. 

Drive cost efficiency and better outcomes: Delegating some or all of your organization’s security efforts to external resources such as MSSPs or managed detection and response (MDR) services offers immediate access to specialized expertise, advanced technologies, 24/7 monitoring, and scalable solutions without the overhead of an in-house team. 

Why partner with a Google Cloud SecOps MSSP?

Choosing a certified Google Cloud MSSP partner means gaining access to differentiated, end-to-end security solutions powered by Google Cloud Security products, including Google Security Operations, Google Threat Intelligence, and Mandiant Solutions. Our tools provide technical advantages like comprehensive data ingestion from multiple clouds and context-aware detections to prioritize threats. 

They can help our partners to deliver unique managed security services offerings tailored to the security requirements of your business. They help you offload security operations strain, increase risk awareness, and significantly reduce response times. They can protect your workloads regardless of location (on-premises or multicloud) and integrate with your existing security investments. 

Hear from some of our partners and customers on the value they are seeing:

“In partnership with Google Cloud, we ensure comprehensive protection for your workloads, regardless of their location. We leverage telemetry from your existing security solutions to provide seamless and robust defense. This integrated approach maximizes your security investment and minimizes risk,” said Laurent Besset, deputy CEO, Cyberdefense Ops, I-TRACING.

As an example of the results possible on Google Cloud, Jayesh Barai, VP of Sales at Netenrich, shared a recent customer success story: “We’ve seen customers achieve transformative results by tackling legacy security operations head-on. With Netenrich’s AI-driven Adaptive MDR solution, one client’s security efficacy became remarkable: they reduced their mean time to detect and respond from hours to just 15 minutes and cut monthly security incidents needing manual intervention from nearly 2,000 to fewer than 10. This operational efficiency also drove major cost savings, including a 50% reduction in annual security expenses and an 80% reduction in SOC staffing requirements, dramatically streamlining their ability to integrate new acquisitions in days instead of months.”

“Google Cloud’s global reach and unwavering commitment to security and innovation make it an ideal partner to help us safeguard our clients’ most valuable assets. Google Cloud’s experience and expertise in the field, coupled with its transparent and open approach, instill a level of trust that is essential in today’s interconnected world.” said Scott Goree, senior vice-president, Partners, Alliances, and Ecosystems, Optiv.

Find the right partner

You can learn more about how a Google Cloud MSSP partner can help your organization modernize security operations by visiting our updated MSSP Page.

AI has accelerated startup innovation more than any technology since perhaps the internet itself, and we’ve been fortunate to have a front row seat to much of this innovation here at Google Cloud. Nine of the top ten AI labs use Google Cloud, as does nearly every AI unicorn and more than 60% of the world’s gen AI startups overall.

And the trend of AI startups choosing us is accelerating. For example, we’ve seen more than a 20% increase over the past year in the number of newer AI Startups (those who recently raised a Series A or B round) choosing to use Google Cloud.

These startups are building with our differentiated AI stack, which offers leading technology and choice at every layer, including a variety of TPU and GPU specialized chips for compute, and a broad range of models that excel at coding, image and video generation, collaboration, and more.

Today, we’re hosting our first-ever Google’s AI Builders Forum, bringing hundreds of startup founders, and builders together in the heart of Silicon Valley to hear about how Google’s AI, infrastructure, and services, as well as our unique resources to support startups’ growth, can help them build.

We’re also excited to announce momentum with dozens of startup customers who are building platforms, products, and entire businesses on Google Cloud, including:

• Afooga, an AI-powered content experimentation factory, enables businesses to generate, test, and distribute content at massive scale from a single hypothesis, automatically optimizing across TikTok, Meta, YouTube, and more. Afooga now leverages Vertex AI and Veo for generative video capabilities.

• Anara, a generative AI research assistant, helps users find and understand scientific documents with verifiable AI summaries and insights. It now uses Google Cloud’s scalable infrastructure, AI Studio, and Cloud Functions to power its models and data processing for a global user base.

• Aptori, an AI security company, detects vulnerabilities in AI-generated code, prioritizes risks, and automates code fixes in real-time. Now, Aptori uses Gemini to analyze code for security weaknesses and generate context-aware fixes, integrating its AI Security Engineer directly into developer workflows.

• aSim, an AI-powered mobile app development tool, allows people to quickly generate, share, and discover mini-apps. Users can now instantly generate an app from a prompt using Gemini models and multimodal capabilities from models like Gemini 2.5 Flash Image (i.e. Nano Banana) and Veo 3.

• CerebraAI develops AI software for analyzing non-contrast CT scans (NCCT), with a focus on early stroke and cancer detection. It fine-tunes MedGemma on NCCT images and leverages Gemini’s few-shot generalization capabilities to rapidly adapt its model for various diagnostic tasks.

• Clavata.ai delivers an integrated AI governance and safety platform with multi-modal, real-time evaluations powered by Gemini models. Its suite of tools enable proactive policy enforcement, dynamic debugging, iteration, observability, and problem prevention.

• Clip Media built Reclip, a social media application that leverages Google’s generative media models like Veo and Imagen to create short, engaging animated videos from real-time audio clips, captured by their proprietary app.

• CoVet is an AI assistant built for veterinary professionals, that uses Gemini, Cloud Functions, and other Google Cloud solutions to help veterinary teams automate administrative work, save hours every day, and refocus on what matters most: exceptional patient care.

• ColomboAI has built its next-generation social engagement super-app on Google Cloud, integrating search, social, commerce, and news into one experience. The company is now launching CAIRO, its in-app AI Operator agent that can automate actions across modules — from GenAI to feeds, shop, and beyond — powered by Google Cloud AI infrastructure.

• Corma is using Google Cloud’s AI infrastructure to train novel, domain-specific foundation models.

• Factory, a platform for agent-driven software development, accelerates engineering by unifying context from sources like GitHub and Jira to delegate tasks such as feature development and migrations. It uses Gemini 2.5 Flash for data ingestion and Gemini 2.5 Pro for advanced code and document generation.

• Gobii provides AI Agents that automate complex web tasks like forms and workflows directly in the browser. To power these intelligent agents, Gobii is building on Vertex AI and scalable GKE infrastructure.

• InstaLILY is using Gemini 2.5 and Vertex AI to power AI agents, called InstaWorkers, that can support sales, service, and operations. 

• Inworld AI, an AI platform for builders of consumer applications, uses Google Cloud and Gemini to cost-effectively handle tens of millions of concurrent users with response times measured in milliseconds, meeting strict requirements for quality, cost control, and security.

• Krea.ai, a creative suite of AI tools, offers real-time image and video generation and personalized model training for artists and marketers. It integrates with Google Cloud to provide access to advanced models like Veo3 and Gemini 2.5 Flash Image (Nano Banana), enabling users to create high-quality ads, product photos, and game assets.

• Lovable, which is building an AI software engineer, uses Vertex AI to deploy and unify its core language models, including Gemini and Anthropic’s models. This unique orchestration enables users to create complete, full-stack web applications from plain English descriptions while choosing the model on Google Cloud that best suits their needs.

• MaestroQA, a conversation analytics data platform, is now using Gemini to enhance its AI-powered conversation analytics. Gemini helps MaestroQA improve its ability to analyze customer interactions across every channel, providing deeper insights that help businesses boost customer satisfaction and drive growth and retention.

• Markups.ai created Agent Marko, an AI contract negotiation agent, which helps turn days-long human legal review into an easier automated process. Users simply email contracts to a designated inbox and receive customized revisions and analysis back within minutes. Gemini 2.5 Pro has enabled Markups.ai to go from handling only first revisions of NDAs to effectively any revision of any contract.

• MLtwist, an AI data pipeline services company, processes, transcribes, translates, and labels large, complex data streams for enterprise applications. It now uses Gemini models and AI Studio for transcription and labeling tasks.

• MNTN uses Google Cloud to power its connected-TV ad platform which helps users better measure the impact of their TV campaigns. MNTN uses Google’s Security Command Center to ensure users have a secure experience on its platform.  

• Mosaic lets users build and run multimodal video-editing AI agents. It uses Gemini 2.5 Pro, Cloud Storage, and Cloud Run to power its platform.

• OpenArt, an AI video production company, empowers social media creators and SMBs to turn ideas into videos complete with motion, music, and a narrative arc with Gemini models and Veo3.

• OpenEvidence is building its AI-powered medical search platform on Google Cloud, enabling healthcare professionals to access medical information more quickly, safely, and accurately.

• Owl.AI, Owl.AI, a sports technology company, delivers AI-powered solutions to professional sports leagues. Their offerings, which include judging and scoring, aim to enhance accuracy, consistency, and eliminate bias. Owl.AI achieves this by leveraging AI models built on Gemini and fine-tuned on Google Cloud to analyze real-time video footage of athletic performances.

• Parallel uses Gemini models via Vertex AI to orchestrate “micro agents” that can manage AI search requests more effectively.

• Prodia, a generative AI API, offers developers APIs to integrate generative AI into their creative tools. Prodia relies on GPUs on Google Cloud to serve text-to-image and instruct-to-edit models, our low-latency network to ensure scalable, cost-effective performance, Dynamic Workload Scheduler for burst capacity, and models like Veo 3 and Gemini 2.5 Flash Image to power advanced video and image generation.

• Provenbase has built its talent recruitment tool for businesses on Google Cloud and is now powering its transformative deep search for talent feature using Google Cloud AI.

• Qualia Clear is an agentic system that improves real estate closings by automating manual title and escrow workflows. It uses tool calling, Gemini 2.5 Flash and Gemini 2.5 Pro, and Google Agent Development Kit to process emails and documents and simplify reporting, improving efficiency and customer service.

• Rembrand is an AI-powered advertising platform that facilitates in-video product placements for content creators and advertisers on social media and connected TV. Powered by Google Cloud’s AI infrastructure, Rembrand enables brands to genuinely connect with audiences without disrupting the content.

• Resolve AI, an always-on AI SRE, autonomously investigates incidents and helps run production systems using code, infrastructure, and observability data. With the intelligence and performance of Gemini on Vertex AI, Resolve AI improves MTTR, reliability, and engineering velocity for its customers.

• SandboxAQ is scaling its AI-accelerated drug discovery and materials design platform on Google Cloud by training Large Quantitative Models (LQMs) on high-accuracy, physics-based simulations run across massively parallel compute nodes on Google Cloud. SandboxAQ recently made two large-scale datasets, called SAIR and AQCat25, publicly available for building and benchmarking LQMs for structure-aware drug potency prediction and for catalyst design, respectively.

• Satlyt, a space compute company, enables in-orbit AI workloads by orchestrating intersatellite communication and routing. It uses Google Kubernetes Engine, Vertex AI, and scalable data infrastructure to deploy AI agents and plans to deploy Google’s Gemma models in orbit.

• Satisfi Labs will begin using Gemini models to power a new agentic platform for hundreds of customers in sports, entertainment, and tourism. The Satisfi AI platform will offer specialized agents supporting guest experiences, ticketing, on-site safety, and merchandise. Tuned by industry experts, Satisfi AI helps live experience businesses sell more, service faster, and gain real-time insights.

• Savvy, an education-focused AI startup, revolutionizes learning by automatically generating flashcards and quizzes from PDFs, notes, videos, and podcasts. As students answer, Savvy uses Gemini to dynamically grade their answers, providing instant feedback and personalized learning.

• Skyvern helps companies automate browser-based workflows with AI. Skyvern uses models like Gemini 2.5 Pro and computer vision to interact with websites, enabling it to automate tasks like filling out forms, procuring materials, and downloading invoices. 

• Subject.com, an AI-powered platform for grades 6 to 12, blends cinematic storytelling with advanced AI to make learning more engaging. Vertex AI, CloudSQL, and BigQuery power Subject’s teacher assistant tool, Spark, that provides instant feedback, an ‘Explain This’ text simplifier, a 24/7 Homework Helper, and personalized learning tied to student interests.

• Tali.ai is a medical AI scribe designed to reduce the administrative burden on clinicians. Integrated with multiple electronic medical record systems across the U.S. and Canada, the company uses Vertex AI and Gemini models to automate clinical note-taking during patient visits and extract key insights.

• Tinuiti, a performance marketing agency, used Vertex AI to develop an AI-powered service that develops and optimizes ad copy to increase performance.

• Toonsutra, an India-based webcomic platform, is using Google Gemini to go global. By making stories accessible in regional languages and adding Lyria 2 for music, Gemini for voices, and Veo 3 for animation, the company is creating next-generation immersive comics for more audiences.

• turbopuffer offers serverless vector and full-text search, helping AI businesses overcome the high costs and complexity of traditional database architectures. ​Its solution, built on Google Cloud infrastructure, has reduced AI database cost by up to 90% for customers, manages more 1 trillion documents, and handles more than 10 million writes and 10,000 queries every second.

• Upwork, a human and AI-powered work marketplace, connects businesses with independent professionals. By leveraging the Vertex AI Text-to-Speech API, Upwork now delivers faster, more accurate talent matching and improved hiring efficiency for clients and freelancers.

• Visla is an AI-powered video creation platform that helps businesses and creators produce pro videos in minutes. Using Imagen 4, Gemini Flash Image 2.5, Veo 3, and Visla’s AI Video Agent with Avatars, it adapts visuals and narration while automating polished content creation for learning, training, and marketing.

• Windsurf, a leader in AI-assisted coding, is using Gemini 2.5 Pro to power its coding assistance IDE, as well as using Gemini models to support integrations with Cognition’s Devin AI.

• Zapia AI, a retail technology company, uses AI agents to support millions of users with product discovery, local business searches, and purchase assistance, resulting in over 90% positive user feedback. Its multi-agent orchestration is powered by Gemini to improve agent reasoning, reduce latency, and lower operational costs.

• Zefr supports Fortune 500 brand advertising with safety and suitability on platforms like YouTube and TikTok, using Gemini models and Vertex AI to analyze video, image, audio, and text.

Google Cloud’s differentiated AI stack for startups

Around the world, startups are using Google Cloud at every step of development and every stage of growth. They’re using our AI and infrastructure to train and serve models, to handle multimodal requests, to build and scale applications, to develop AI agents, to manage underlying data, to support developers, to reach customers through our marketplace, and more.

Whatever the use case, our full AI-optimized tech stack provides startups with leading technology and choice at every layer, including:

• Highly-performant first-party chips (TPUs) and third-party chips (GPUs) for training and inferencing, as well as the ability to move workloads between the two.

• AI Studio, a fast way for startups to get started building applications with our models.

• Our unified AI development platform, Vertex AI.

• More than 200 AI foundation models, including our leading multimodal models like Nano Banana and Veo 3, and popular third-party models like Anthropic’s Claude and the open-source Llama and Mystral.

• Developer tools like Gemini CLI, which provides access to Gemini directly within the coding terminal, streamlining workflows and boosting productivity.

• Products like BigQuery and AlloyDB to handle an organization’s underlying data.

• Security operations and threat intelligence tools to help startups secure their applications.

• Services like Google Kubernetes Engine, Cloud Storage, and CloudSQL to quickly scale workloads and applications up and down.

If you can dream it, then you can build it, launch it, scale it, and sell it with Google Cloud.

More programs, initiatives, and product access for startups

One of the biggest challenges to starting up isn’t having the idea or the right technology — it’s access to the resources to make it happen and expertise to help tackle any challenges that arise.

Google Cloud provides a unique set of resources and benefits for startups through our Google for Startups Cloud Program, including cloud credits that are applicable across all Google Cloud products and AI models — including those from our partners like Anthropic and Meta. The program also offers mentorship and go-to-market programs that help startups reach customers, and even on-the-ground engineers who engage with and support startups as they build.

We also engage deeply with the startup community through our global series of Accelerators and our partnerships with top venture capital firms, incubators, and community organizations — making it easier than ever for startups to access Google Cloud’s unique technology and our extensive set of resources to help startups build and market their products.

For example, AI startups who work with firms like Lightspeed, Sequoia, or Y Combinator all have preferred access to our Google for Startups Cloud Program — supercharging their work with up to $350,000 in cloud credits along with mentorship and engineering support.

We’re committed to making it as easy as possible for startups to get building with Google Cloud AI. If you’re interested in our technology and want to get started, head to Google AI Studio or our Google for Startups Cloud Program and let’s get building together!

As enterprises increasingly adopt model context protocol (MCP) to extend capabilities of AI models to better integrate with external tools, databases, and APIs, it becomes even more important to ensure secure MCP deployment. 

MCP unlocks new capabilities for AI systems; it can also introduce new risks, such as tool poisoning, prompt injection, and dynamic tool manipulation. These can lead to data exfiltration, identity subversion and misuse of AI systems. 

Securing an MCP deployment begins with a strong security foundation. Here are five key MCP deployment risks you should be aware of, and how using a centralized proxy architecture on Google Cloud can help mitigate them. 

Top five MCP deployment risks you should know

While there are some broader risks unique to AI, these five are especially important to be aware of when designing MCP deployments:

1. Unauthorized tool exposure: A misconfigured MCP manifest can create a vulnerability that allows unauthorized individuals or agents to access sensitive tools, such as internal administration functions.  

2. Session hijacking: An attacker may steal a legitimate user’s session ID to impersonate them. This allows the attacker to either make unauthorized API calls directly or, in stateful systems, inject malicious payloads into a shared data queue for delivery to the victim’s active session.

3. Tool shadowing and shadow MCP: Rogue MCP tools, mimicking legitimate services, can be deployed by malicious actors. This deceptive practice can trick both AI agents and human employees into interacting with harmful tools under the belief they are genuine.

4. Sensitive data exposure and token theft: Improperly configured environments or inadequate data-handling practices can lead to the accidental exposure of sensitive information, such as API keys, credentials, tokens, and personally identifiable information (PII). Attackers can exploit exposed credentials to gain unauthorized access to corporate resources, which could lead to significant data breaches.

5. Authentication bypass: Weak, misconfigured, or inadequately-enforced authentication mechanisms can be exploited by attackers to circumvent security controls, allowing them to gain unauthorized access by successfully impersonating legitimate users and trusted entities.

How a centralized MCP proxy architecture can help

To address these core risks in a scalable manner, we recommend a pattern built around a centralized MCP proxy that acts as a secure intermediary for all communication between clients and MCP servers.

Built on Cloud Run, Apigee, or Google Kubernetes Engine, this intelligent proxy intercepts all tool calls, enforcing your organization’s security policies and providing extensive monitoring. By serving as a centralized security enforcement point, the MCP proxy enables important risk-managing capabilities such as consistent access controls (acts as an authorization server), advanced traffic management, audit logging, secret scanning, resource limits, and real-time threat detection, all without requiring modifications to individual remote MCP servers.

Without centralization, organizations deploying MCP servers face challenges:

• Fragmented authentication: Each MCP server enforces its own policies, creating inconsistencies.

• Operational overhead: Updating agents with new endpoints is extensive and error-prone.

• Security blind spot: Limited visibility into health, usage, and anomalies across distributed MCP servers.

• Expended attack surface: Vulnerabilities such as prompt injection, tool poisoning, and unmonitored traffic flows.

A unified MCP proxy on Google Cloud addresses these challenges by:

• Enforcing organizational security policies at a single, managed entry point for all MCP requests and responses.

• Authenticating and authorizing consistently using the Google Identity Platform.

• Centralizing observability through Cloud Logging and Security Command Center.

• Standardizing protections against threats, aligned with Google secure AI agent guide and  OWASP guidance for agentic AI applications.

Architecture overview

The following reference architecture shows how Google Cloud services help secure MCP client requests end-to-end.

Key components:

• Global Load Balancing provides a single entry point for all MCP client traffic.

• Certificate Manager manages TLS certificates for secure communication.

• Cloud Armor provides DDoS and WAF protections against common OWASP web threats.

• MCP Proxy acts as the central enforcement layer for authentication, authorization, request inspection, and routing.

• Google Identity Platform validates identities and issues OAuth 2.0 tokens.

• Model Armor screens model prompts and responses for prompt injection and jailbreak detection, sensitive data, and can help ensure responsible AI practices.

• Secret Manager stores API keys, database credentials, and sensitive configuration values.

• Artifact Registry ensures MCP server images are scanned, verified and encrypted at rest with customer managed encryption keys.

• Internal Load Balancing routes sanitized traffic to the appropriate MCP servers.

• Cloud Logging and Security Command Center deliver visibility, anomaly detection, and compliance reporting.

Layered security controls

Google Cloud applies security at every layer of the architecture. Let’s dig into these capabilities below.

Network-level security control

• Network segmentation can isolate MCP tools servers, load balancers, and databases using virtual private networks (VPC), Cloud Run ingress, and Cloud Run egress options to reduce lateral movement risk. 

• Web attack and DDOS protection can help block malicious traffic, enforce IP allow and deny lists, and provide resilience against DDoS attacks with Global Load Balancer with Cloud Armor.

• Advanced traffic management can help provide custom domain path matching for MCP tool servers, using Internal Application Load Balancer (ALB) with advanced traffic management capabilities. By inspecting the incoming request host and path, the ALB intelligently directs requests to the appropriate backend service or instance of your MCP server. 

The backend service supports various back end types which can be MCP servers deployed in Google Cloud (such as VMs, Cloud Run, and Google Kubernetes Engine), in a remote network (using hybrid NEG) or on the internet (using internet NEG).

Authentication and authorization

The MCP proxy functions as the MCP Authorization Service (MAS) and integrates with an external Identity provider (such as Google Identity Platform, Okta, and Entra ID). This allows for secure access to your MCP server without the need to build and maintain a complex identity system. 

This highly scalable and reliable authentication solution can help users securely authenticate through their existing identity platform. Additionally, Cloud Identity and Access Management (IAM) can enable enforcement of role-based access, and restrict access to cloud resources.

Protection in-line MCP client prompt and response with Model Armor

Proxy leverages Model Armor to provide robust, real-time protection against runtime threats like prompt injection, jailbreaking, tool poisoning, dynamic tool manipulation, and sensitive data leakage. 

Using Model Armor templates, you can configure the MCP client prompts to be scrutinized and sanitized by Model Armor before being forwarded to the application load balancer. This process ensures that only sanitized prompts reach your MCP tools servers, thereby preventing unintended actions.

Logging and auditing

All actions performed by MCP clients will be logged by the proxy to Cloud Logging, such as accessing specific tools, modifying or writing to memory locations, initiating or receiving communication. The log will include a timestamp, agent ID, session ID, payload, and input/output signature fingerprints. 

The comprehensive nature of these logs, combined with the detailed metadata, can help administrators and auditors collect the necessary information to reconstruct client activities, identify anomalies, and enforce security policies effectively.

Additional security controls 

In addition to layered security controls, a robust defense-in-depth strategy requires continuous monitoring and proactive measures to identify and mitigate threats.

Secrets scanning with Sensitive Data Protection

Sensitive Data Protection (SDP) discovery periodically scans the MCP Tools server on Cloud Run, specifically targeting the detection of hardcoded secrets. This proactive measure involves regular scans of the server’s build and runtime environment variables. 

These scans detect insecure coding-practice risks, such as instances where sensitive information (including API keys, database credentials, and access tokens) has been inadvertently embedded directly in the code or configuration.

Vulnerability scanning

MCP server images stored in Artifact Registry are scanned for vulnerabilities before they are deployed. When a new image is pushed to Artifact Registry, it is automatically scanned for known vulnerabilities, providing actionable insights into potential weaknesses.

You can enforce policies that block the deployment of images with critical vulnerabilities as part of securing your environment from the ground up.

Threat detection

Security Command Center (SCC) provides AI protection that helps manage the security posture of your AI workloads by detecting threats and helping with  mitigating risks to AI asset inventories. This includes managing threats to the MCP deployment through detection, investigation, and response. 

SCC can also identify unauthorized access and data exfiltration attempts, delivering real-time alerts and remediation recommendations.

AI Protection helps you manage the security posture of your AI workloads by detecting threats and helping you to mitigate risks to your AI asset inventory.

Bring it all together

By strategically implementing these Google Cloud security services, you can establish a secure and resilient environment for your Model Context Protocol remote servers. Prioritizing authentication, network security, data protection, and continuous monitoring will ensure the integrity and confidentiality of your AI models and the sensitive information they process. 

This unified, Zero Trust approach can help mitigate emerging AI-specific risks, and also provide a scalable and future-proof foundation for the evolution of your AI-driven applications.

To learn more about securing your AI workload, please refer to our documentation.

MCP Toolbox for Databases (Toolbox) is an open-source MCP server that makes it easy for developers to connect gen AI agents to enterprise data, with initial support for databases like BigQuery, AlloyDB, Cloud SQL, and Spanner. Since launching earlier this year, Toolbox has made it easier for millions of developers to access enterprise data in databases.

Today, we’re expanding Toolbox with a comprehensive new set of tools for Firestore. This will help developers build more modern web and mobile applications. Let’s explore how these new capabilities can improve your development process.

What MCP is, and how it unlocks AI-assisted workflows

MCP is an emerging open standard for connecting AI systems with tools and data sources through a standardized protocol, replacing fragmented, custom integrations. Think of MCP as a universal adapter for AI, allowing any compatible assistant to plug into any tool or database without needing a custom-built connector each time. Now with the MCP Toolbox, these assistants (such as those in an IDE or a CLI like the Gemini CLI) can connect directly to your Firestore database.

This is a massive step for AI-assisted workflows, from debugging data and testing security rules to managing your collections—all using natural language. For instance, a developer building a retail app can now ask their assistant, ‘Find all users whose wishlists contain discontinued product IDs,’ to perform data cleanup, without writing a single line of code.

Let’s explore how these new capabilities can improve your development process.

AI-assisted development meets the NoSQL world

As you carry out AI-assisted tasks, you’re probably looking for the most efficient way to interact with your data. Our new pre-built tools for Firestore enable you to do just that, directly from your Gemini CLI or other AI-powered development environment.

Firestore’s flexible document structure and powerful security rules offer unique capabilities for building modern mobile and web applications. These tools are crafted to empower the Firestore developer, helping them master both the flexibility of the document model and the creation of robust access controls that protect their app. You can now use your AI assistant to perform queries, carry out targeted document updates, and even validate your security rules before you deploy them, saving you time and preventing errors.

From QA bug to resilient feature: A developer’s story

Let’s take a hypothetical example. Alex is a full-stack developer on a team building a new e-commerce application using Firestore. She uses  Gemini CLI to help her code, debug, and test. This morning, a high-priority bug was filed by the QA team: an issue in the staging environment is causing items to reappear in a user’s “wishlist” after being removed. Because of the blog, her release was blocked.

The bug hunt begins

Until now, investigating a bug meant Alex would have to manually click through the Cloud Console to inspect test documents or write a custom script just to query the staging database—a slow and cumbersome process. Now, she can simply have a conversation with Gemini CLI.

The bug report contains the test user accounts. Alex opens her terminal and asks:

“Hey, show me the Firestore data for the test users qa_user_123 and qa_user_456 from the users-staging collection.”

Gemini CLI understands this, calls the firestore-get-documents tool, and instantly displays the JSON for both user documents. Alex confirms the bug—the wishlist array contains stale data. She continues the conversation to understand the scope:

“Okay, that’s the bug. Find all users in the users-staging collection whose wishlist contains product-glasses(inactive).”

CLI uses the firestore-query-collection tool and reports back that 20 test accounts are affected. After developing a code fix, she needs to clean the test environment to verify it.

“For all 20 test users you just found, please remove product-glasses(inactive) from their wishlist.”

Gemini CLI confirms the plan and uses the firestore-update-document tool to perform the cleanup, clearing the way for a successful re-test.

From reactive fix to proactive hardening

The immediate bug is fixed, but Alex wants to ensure this class of error can never happen again. She decides to enforce the correct data structure with Firestore Security Rules.

Until now, validating security rules meant a disruptive context switch. Alex would have to copy her rules, navigate away from her terminal to the Firebase Console’s Rules Playground, or set up a local emulator just to check for syntax errors. This friction often discourages thorough, iterative testing within the main development loop.

Now, Alex drafts her new, stricter security rule. Before deploying, she asks Gemini CLI for a pre-flight check right from her terminal:

“new_rules.txt is a new Firestore Security Rule I’m working on for staging. Can you validate it for me?” 

CLI uses the firestore-validate-rules tool and replies, “The issue is a missing semicolon at the end of the return statement” Alex fixes the typo instantly. For a final check, she asks:

“Show me the active Firestore security rules for this project.”

Using the firestore-get-rules tool,  CLI displays the current ruleset, allowing Alex to do a final comparison. Confident and assured, she deploys her changes.

A task that could have taken hours of manual investigation, scripting, and context switching was completed in minutes. Alex didn’t just fix a bug; she used her AI assistant to make the entire application more resilient.

Getting started

These new Firestore tools within the MCP Toolbox represent our commitment to providing you with powerful, intuitive tools that accelerate the entire development lifecycle. By connecting your Firestore database directly to your AI assistant, you can spend less time on tedious tasks and more time building incredible applications.

Learn more about MCP Toolbox for Databases, connect it to your favorite AI-assisted coding platform, and experience the future of AI-accelerated, database-connected software development today.

• Docs: Connecting your AI Assistant to Toolbox
• Quick Start Guide: Using Firestore Tools w/ MCP
• Contribute to MCP Toolbox
• Join our Discord community

People often think of BigQuery in the context of data warehousing and analytics, but it is a crucial part of the AI ecosystem as well. And today, we’re excited to share significant performance improvements to BigQuery that make it even easier to extract insights from your data with generative AI.

 In addition to native model inference where computation takes place entirely in BigQuery, we offer several batch-oriented generative AI capabilities that combine distributed execution in BigQuery with distributed execution with remote LLM inference on Vertex AI, with functions such as: 

1. ML.GENERATE_TEXT to generate text via Gemini, other Google-hosted partner LLMs (e.g., Anthropic Claude, Llama) or any open-source LLMs
2. ML.GENERATE_EMBEDDING to generate text or multimodal embeddings
3. AI.GENERATE_TABLE to generate structured tabular data via LLMs and their constrained decoding capabilities. 

In addition to the above table-valued functions, you can use our row-wise functions such as AI.GENERATE for more convenient SQL query composition. All these functions are compatible with text data in managed tables and unstructured object files, such as images and documents. Thanks to the performance improvements we are unveiling today, users can expect dramatic gains in scalability, reliability, and usability across BigQuery and BigQuery ML:

1. Scalability: Over 100x gain for first-party LLM models (tens of millions of rows per six-hour job), over 30x gain for first-party embedding models (tens to hundreds of millions rows per six-hour job), and added support for Provisioned Throughput quotas. 

2. Reliability: Over 99.99% LLM inference query completion without any row failures; and over 99.99% row-level success rate across all jobs, with the rare per-row failures being easily retriable without failing the query. 

3. Usability: Enhanced user experience by supporting default connections, enabling global endpoints. We also enabled a single place for quota management (in Vertex AI) by automatically retrieving quota from Vertex AI to BigQuery. 

Let’s take a closer look at these improvements.

1. Scalability 

We’ve dramatically improved throughput for our users on pay-as-you-go (PayGo) pricing model: ML.GENERATE_TEXT function throughput on first-party LLMs has increased by over 100x, and ML.GENERATE_EMBEDDING throughput by over 30x. For users requiring even higher performance, we’ve also added support for Vertex AI’s Provisioned Throughput (PT).

Google embeddings LLMs
In BigQuery, each project has a fixed default quota for first-party text embedding models, including the most popular text-embedding-004/005 models. To enhance utilization and scalability, we introduced dynamic token-based batching to pack as many rows as possible into a single request, under the token constraint. Combined with other optimizations, this boosts scalability from 2.7 million to approximately 80 million rows (30x) per six-hour job (based on a default quota of 1500 QPM and 50 tokens per row dataset). You can further increase this capacity by raising your Vertex AI quota to 10,000 QPM without manual approval, which enables embedding over 500 million rows in a six-hour job.

Early access customers such as Faraday are excited for this scalability boost:

“I just did 12,678,259 embeddings in 45 min with BigQuery’s built-in Gemini. That’s about 5000 per second. Try doing that with an HTTP API!” – Seamus Abshere, Co-founder and CTO, Faraday

Google Gemini: PayGo users via dynamic shared quota
Gemini models served via Vertex AI now use dynamic shared quota (DSQ) for pay-as-you-go requests. DSQ provides access to a large, shared pool of resources, with throughput dynamically allocated based on real-time availability and demand across all customers. We rebuilt our remote inference system with adaptive traffic and producer-consumer-based error-retry mechanism. This enabled us to effectively leverage the higher amount of, but less-guaranteed, quota from DSQ. Based on internal benchmarking results, now we can process roughly 10.2 million rows in a six-hour job with gemini-2.0-flash-001, or 9.3 million rows with gemini-2.5-flash, where each row has an average of 500 input and 50 output tokens. Specific numbers depend on factors such as token count and model. See the chart below for more details.

Google Gemini: Dedicated quota via provisioned throughput
While our generative AI inference with Dynamic Shared Quota offers high throughput, it has an inherent upper bound and the potential for quota errors due to its non-guaranteed nature. To overcome these limitations, we’ve added support for Provisioned Throughput from Vertex AI. By purchasing dedicated capacity, you can help ensure consistently high throughput for demanding workloads, and get a reliable and predictable user experience. After purchasing Vertex AI provisioned throughput, you can easily leverage it in BigQuery gen AI queries by setting the “request_type” argument to “dedicated”. 

2. Reliability 

Facing limited and non-guaranteed generative AI inference quotas, we implemented a partial failure mode to allow queries to succeed, even if some rows fail. Via adaptive traffic control and a robust retry mechanism, across all users’ query traffic, we’ve now achieved 1) over 99.99% generative AI queries can finish without a single row failure, and 2) a row success rate of over 99.99%. This greatly enhanced BigQuery gen AI reliability. 

Note that the above row-level success rate is achieved based on independent queries that invoke our generative AI functions, including both table-valued functions and row-wise scalar functions, where the error retry takes place implicitly. If you have even large workloads that see occasional errors you can use this simple SQL script or the Dataform package, which allow you to iteratively retry failed rows to get to a 100% row success rate in almost all cases.

3. Usability 

We have also made Gen AI experience on BigQuery more user-friendly. We’ve streamlined complex workflows like quota management and BQ connection/permission setup, and enabled global endpoint more accessible experiences for users in all regions.

3.1 Default connection for remote model creation
Previously, setting up gen AI remote inference required a series of manual steps for users to configure a BigQuery connection and grant permissions to its service account. To solve this, we launched a new default connection. This feature automatically creates the connection and grants the necessary permissions, eliminating all manual config steps. 

3.2 Global endpoint for first-party models
BigQuery gen AI inference now supports the global endpoint from Vertex AI, in addition to dozens of regional endpoints. This global endpoint provides higher availability across the world, which is particularly useful for smaller regions that may not have immediate access to the latest first-party gen AI models. Users without a data residency requirement for ML processing can use the global endpoint.

The SQL below illustrates how to create a Gemini model with the global endpoint and the BigQuery default connection.

3.3 Automatic quota sync from Vertex AI
BigQuery now automatically syncs with Vertex AI to fetch your quota. Thus, if you have a higher than default quota, BigQuery uses it automatically, without any manual configuration on your part. 

Get started today

Ready to build your next AI application? Start generating text, embedding, or structured tables using gen AI directly in BigQuery today. Dive into our gen AI overview page and its linked documentation for more details. If you have any questions or feedback, reach out to us at bqml-feedback@google.com.

It’s hard to believe it’s been over 10 years since Kubernetes first set sail, fundamentally changing how we build, deploy, and manage applications. Google Cloud was at the forefront of the Kubernetes revolution with Google Kubernetes Engine (GKE), providing a robust, scalable, and cutting-edge platform for your containerized workloads. Since then, Kubernetes has emerged as the preferred platform for workloads such as AI/ML.  

Kubernetes is all about sharing machine resources among the applications and pod networking is essential for the connectivity between workloads and services using the Container Network Interface (CNI).

As we celebrate the 10th year anniversary of GKE, let’s take a look at how we’ve built out network interfaces to provide you with the performant, secure, and flexible pod networking and how we’ve evolved our networking model to support AI workloads with the Kubernetes Network Driver.

Let’s take a look back in time and see how we got there.

2015-2017: Laying the CNI foundation with kubenet

In Kubernetes’s early days, we needed to establish reliable communication between containers. For GKE, we adopted a flat model of IP addressing so that the pods and the node could communicate freely with other resources in the Virtual Private Cloud (VPC) without going through tunnels and gateways. During these formative years, GKE’s early networking models often used kubenet, a basic network plugin. Kubenet provided a straightforward way to get clusters up and running, by creating a bridge on each node and allocating IP addresses to pods from a CIDR range dedicated to that node. While Google Cloud’s network handled routing between nodes, Kubenet was responsible for connecting pods to the node’s network and enabling basic pod-to-pod communication within the node.

During this time, we also introduced route-based clusters, which were based on Google Cloud routes, part of the Andromeda engine that powers all of cloud networking. The routes feature in Andromeda played a crucial role in IP address allocation and routing within the cluster network using VPC routing rules. This required advertising the pod IP ranges between the nodes. 

However, as Kubernetes adoption exploded and applications grew in scale and complexity, we faced challenges around managing IP addresses and achieving high-performance communication directly between pods across different parts of a VPC. This pushed us to develop a networking solution that was more deeply integrated with the underlying cloud network.

2018-2019: Embracing VPC-native networking

To address these evolving needs and integrate with Google Cloud’s powerful networking capabilities, we introduced VPC-native networking for GKE. This marked a significant leap forward for how CNI operates in GKE, with alias IP ranges (the IP ranges that pods use in a node) becoming a cornerstone of the solution. VPC-native networking became the default and recommended approach, helping to increase the scale of the GKE clusters up to 15K nodes. With VPC-native clusters, the GKE CNI plugin ensures that pods receive IP addresses directly from your VPC network — they become first-class citizens on your network.

This shift brought a multitude of benefits:

• Simplified IP management: GKE CNI plugin works with GKE to allocate pod IPs directly from the VPC, making them directly routable and easier to manage alongside your other cloud resources.

• Enhanced security through VPC integration: Because pod IPs are VPC-native, you can apply VPC firewall rules directly to pods. 

• Improved performance and scalability: GKE CNI plugin facilitates direct routing within the VPC, reducing overhead and improving network throughput for pod traffic.

• A foundation for advanced CNI features: VPC-native networking laid the groundwork for more sophisticated CNI functionalities that followed.

We referred to GKE’s implementation of CNI’s with VPC-native networking as GKE standard networking with dataplane v1 (DPv1). During this time, we also announced GA support for network policies with Calico. Network policies allow you to specify rules for traffic flow within your cluster, and also between pods and the outside world.

2020 and beyond: The eBPF revolution

The next major evolution in GKE’s CNI strategy arrived with the power of extended Berkeley Packet Filter or eBPF, which lets you run sandboxed programs in a privileged context. eBPF makes it safe to program the Linux kernel dynamically, opening up new possibilities for networking, security, and observability at the CNI level without having to recompile the kernel.

Recognizing this potential, Google Cloud embraced Cilium, a leading open-source CNI project built on eBPF, to create GKE Dataplane V2 (DPv2). Reaching general availability in May 2021, GKE Dataplane V2 represented a significant leap in GKE’s CNI capabilities: 

• Enhanced performance and scalability: By leveraging eBPF, CNI can bypass traditional kernel networking paths (like kube-proxy’s iptables-based service routing) for highly efficient packet processing for services and network policy.

• Built-in network policy enforcement: GKE Dataplane V2 comes with Kubernetes network policy enforcement out-of-the-box, meaning you don’t need to install or manage a separate CNI like Calico solely for policy enforcement when using DPv2.

• Enhanced observability at the data plane layer: eBPF enables deep insights into network flows directly from the kernel. GKE Dataplane V2 provides the foundation for features like network policy logging, offering visibility into CNI-level decisions.

• Integrated security in the dataplane: eBPF enforces network policies efficiently and with context-awareness directly within CNI’s dataplane.

• Simplified operations: As it’s a Google-managed CNI component, GKE Dataplane V2 simplifies operations for Customer workloads.

• Advanced networking capabilities: Dataplane V2 unlocks a suite of powerful features that were not available or harder to achieve with Data Plane V1. These include:

• IPv6 and dual-stack support: Enabling pods and services to operate with both IPv4 and IPv6 addresses.

• Multi-networking: Allowing pods to have multiple network interfaces, connecting to different VPC networks or specialized network attachments, crucial for use cases like cloud native network functions (CNFs) and traffic isolation.

• Service steering: Providing fine-grained control over traffic flow by directing specific traffic through a chain of service functions (like virtual firewalls or inspection points) within the cluster.

• Persistent IP addresses for pods: In conjunction with the Gateway API, GKE Dataplane V2 allows pods to retain the same IP address across restarts or rescheduling, which is vital for certain stateful applications or network functions.

GKE Dataplane V2 is now the default CNI for new clusters in GKE Autopilot mode and our recommended choice for GKE Standard clusters, underscoring our commitment to providing a cutting-edge, eBPF-powered network interface.

2024: Scaling new heights for AI Training and Inference

In 2024, we marked a monumental achievement in GKE’s scalability, with the announcement that GKE supports clusters of up to 65,000 nodes. This incredible feat, a significant jump from previous limits, was made possible in large part by GKE Dataplane V2’s robust, highly optimized architecture. Powering such massive clusters, especially for demanding AI/ML training and inference workloads, requires a dataplane that is not only performant but also incredibly efficient at scale. The version of GKE Dataplane V2 underpinning these 65,000-node clusters is specifically enhanced for extreme scale and the unique performance characteristics of large-scale AI/ML applications — a testament to CNI’s ability to push the boundaries of what’s possible in cloud-native computing.

For AI/ML workloads, GKE Dataplane v2 also supports ever-increasing bandwidth requirements such as in our recently announced A4 instance. GKE Dataplane v2 also supports a variety of compute and AI/ML accelerators such the latest GB200 GPUs and Ironwood, Trillium TPUs.

For today’s AI/ML workloads networking plays critical role: AI and machine learning workloads are pushing the boundaries of computing as well as networking, presenting unique challenges for GKE networking interfaces:

• Extreme throughput: Training large models requires processing massive datasets that demand upwards of terabit networking orchestrated by GKE networking interfaces.

• Ultra-low latency: Distributed training relies on near-instantaneous communication between processing units.

• Multi-NIC capabilities: Providing pods with multiple network interfaces, managed by GKE Dataplane V2’s multi-networking capability, can significantly boost bandwidth and allow for network segmentation.

2025 – Beyond CNI: addressing next gen Pod Networking challenges

Dynamic resource allocation (DRA) for networking 

A promising Kubernetes innovation is dynamic resource allocation (DRA). Introduced to provide a more flexible and extensible way for workloads to request and consume resources beyond CPU and memory, DRA is poised to significantly impact how CNIs manage and expose network resources. While initially focused on resources like GPUs, its framework is designed for broader applicability.

In GKE, DRA (available in preview from GKE version 1.32.1-gke.1489001+) opens up possibilities for more efficient and tailored network resource management, helping demanding applications get the network performance they need using the Kubernetes Network Drivers (KNDs).

KNDs use DRA to expose Network resources at the Node level that can be referenced by all the Pod (or all containers). This is particularly relevant for AI/ML workloads, which often require very specific networking capabilities. 

Looking ahead: Innovations shaping the future

The journey doesn’t stop here. With the increased adoption of accelerated workloads driving new architectures on GKE, the demands on Kubernetes networking will continue. One of the reference implementations for the Kubernetes Network Driver is the DRANET project. We look forward to continued discussions with the community and contributions to the DRANET project. We are committed to working with the community to deliver innovative customer centric solutions addressing these new challenges.

Trading in capital markets demands peak compute performance, with every microsecond impacting critical decisions and market outcomes. At Google Cloud, we’re committed to providing global markets with the cutting-edge infrastructure they need to create and participate in digital exchange ecosystems. Our industry investments enable a purpose-built, cloud-native market infrastructure solution leveraging a global network that was built for security and scale, data, and AI capabilities. At the same time, we’re building industry-specific innovations that offer performant, scalable, and resilient environments for exchanges and trading participants, ultimately transforming how they access markets, utilize data, and manage risk. 

Following our investments in optimized infrastructure for digital exchanges, we’ve continued to push the boundaries of what’s possible in cloud-based trading.

The general availability of our C3 and C4 machine series, powered by the latest Intel Xeon Scalable processors and our custom Titanium offload network investments, represents a significant leap forward for latency-sensitive trading applications. In addition, Citadel Securities recently joined us as part of our Cloud WAN layer 2 solution that enables point-to-point connections over Google’s proprietary global network and complements our existing Network Connectivity Center layer 3 solution. Together, these offerings help trading participants achieve low-latency compute and connectivity for their globally distributed trading infrastructure. 

Building on these foundational investments, we’re excited to announce new benchmarks specifically tailored for trading participants who require minimal latency and jitter with maximal throughput to handle the increasing market velocity of their trading infrastructures. In collaboration with 28Stone, a consultancy with expertise in capital markets technology and electronic trading solutions, we’ve tested and validated the performance of our C3 machine types to meet the needs of trading participants.  

28Stone’s published report highlights that Google Cloud can achieve a round-trip trading decision in less than 50 microseconds at P99 across a range of compute profiles.

“As a 24×7 digital exchange focused on delivering institutional exchange level consistency in the cloud, we are excited to see that Bullish’s participants can immediately leverage what 28Stone and Google Cloud have publicly demonstrated: the technical capabilities to rapidly spot opportunity and engage with confidence.” – Alan Fraser, VP of Platform & Operations, Bullish

The role of latency, jitter, and throughput

In today’s hyper-competitive electronic trading landscape, the performance of underlying technology infrastructure is not just a contributing factor to success — it’s fundamental. Three key performance metrics stand out for their impact on trading outcomes: latency, jitter, and throughput. 

Latency: the race 

In the context of trading, latency refers to the time it takes to receive, understand, and decide an action from a single datum. For trading systems, this means the time it takes for market data to reach the trading algorithm, for that algorithm to make a decision, and finally send a response back to the exchange. In a world of high-frequency trading (HFT) and algorithmic execution, single-microsecond delays can mean the difference between a profitable trade and a missed opportunity, a less favorable execution price, or getting filled on a resting order.

28Stone demonstrated latency for participants to make a straightforward trade decision — from receiving the ticks to processing a trade decision — of between 1.5µs and 3.5µs (P50 and P99, respectively) for normal replay speed of CME Group Equity market pcap files using open source Data Plane Development Kit (DPDK) network acceleration. Additionally, they demonstrated similar latency profiles with data rates increasing by up to 100x.

Jitter: the quest for consistency 

Jitter refers to the variation in latency over time. While low latency is critical, high jitter — meaning unpredictable and inconsistent delays — can negatively impact trading performance. If the time it takes for an order to reach the exchange varies significantly, it becomes incredibly difficult to predict execution outcomes, manage risk, or implement trading strategies that rely on time certainty. If you were to use UberEats to order lunch but were told it would be delivered between 12 and 5pm, you would be unlikely to choose that option.  

28Stone demonstrated that participants can expect their experience in Google Cloud to be uniform, regardless of volatile market conditions. Google Cloud infrastructure delivers low jitter, allowing trading algorithms to operate with a higher degree of certainty, leading to more stable and reliable market mechanics – as well as profitability from good trading signals. 

As illustrated below for various C3 instance sizes using C++ with or without DPDK, the percentiles demonstrate strong consistent message performance. The increase in latency between each percentile demonstrates network and compute consistency measured in single and low tens of microseconds. The 28Stone report contains complete histograms for various configurations, allowing customers to see how to balance their specific latency and jitter requirements against the configurations’ cost profiles.

Throughput: handling the pressure of information

Throughput measures the amount of data that can be processed or transmitted within a given window of time (typically one second). In trading, throughput is a system’s capacity to handle large volumes of market data updates, process numerous events simultaneously, and aggregate trade events efficiently — especially during periods of high market volatility or peak trading hours. Insufficient throughput can lead to data queues, order rejections, increased risk exposure, and an inability to keep pace with market activity.

The C3 machine series leverages Google Cloud’s high-bandwidth networking capabilities, which include up to 200 Gbps per VM Tier 1 networking and services such as Cloud WAN. These machines are designed to provide the high throughput traders need to ingest vast streams of market data and execute on a large number of orders per second. The result is a trading system that performs optimally under strenuous load.

As highlighted, 28Stone tested increasingly higher replay speeds, resulting in higher bit rates that various trading systems may see in a variety of markets.

In summary, minimizing latency, reducing jitter, and maximizing throughput aren’t abstract technical pursuits but about consistency and certainty, akin to your lunch order arriving near your lunch break and not in the evening. Modern capital markets trading participants and digital exchanges demand these capabilities to enable market quality, fairness, and operational resilience. 

Embracing new market dynamics with cloud

Financial markets are in a perpetual state of flux, characterized by evolving regulations, a proliferation of new asset classes (including those that trade 24/7 like FX and digital assets), and sudden, sharp spikes in trading volumes. In this dynamic environment, the ability to scale infrastructure rapidly, operate with flexibility, and manage costs effectively is not just an advantage — it’s a prerequisite for survival and growth. 

With Google Cloud, exchanges and participants enjoy several benefits:  

• Elastic scalability: Automatically scale resources up or down based on real-time demand. This means that during unexpected volume spikes — driven by market news, geopolitical events, or algorithmic trading activity — trading infrastructure can dynamically access additional compute power to maintain optimal performance. When volumes normalize, resources can be scaled back down, so that firms only pay for what they use. 

• Flexibility for continuous trading: Cloud infrastructure provides the resilience and availability that continuous 24/7 trading requires, as demonstrated daily by critical workloads for global 24/7 industry platforms like air travel, retail banking, media, and retail. Google Cloud’s global network and multiple regions help ensure high uptime and fault tolerance, both critical for markets that never sleep. As seen with Deutsche Börse’s development of a digital asset trading platform on Google Cloud, the architecture is designed for 24/7 availability and can be rolled out quickly to new markets. Google Cloud’s continuous operations allow firms to capitalize on opportunities around the clock without the massive investment and operational overhead of maintaining private data centers with equivalent N+1 redundancy globally.

• Rapid engagement in new markets: Historically, expanding into new geographical markets or launching new asset classes involved lengthy and costly infrastructure build-outs. With Google Cloud’s global regions, firms can deploy trading infrastructure in new regions in a fraction of the time. This agility allows for rapid market entry, thereby enabling businesses to seize new revenue streams and diversify their business with lower upfront investment costs and risk. Quickly provisioning and de-provisioning resources also means firms can experiment with new market strategies more freely, knowing they are not locked into long-term hardware commitments.

Want to see it for yourself?

By harnessing the scale, flexibility, and compelling cost-to-performance ratio of Google Cloud, including the powerful C3 and C4 family instances, trading participants can transform market volatility from a threat into an opportunity. They can confidently handle volume surges, support round-the-clock trading, and swiftly enter new markets, all while maintaining tight control over their operational expenditure and maximizing their competitive edge.

Want to apply these capabilities to the announced CME Group market migration to Google Cloud? Register with Google Cloud for notifications and engagements.

The Domain Name System (DNS) is like the internet’s phone book, automatically and near-instantly translating requests for websites and mobile apps from their domain names to the Internet Protocol addresses of the actual computers hosting them. As part of the bedrock of the internet, DNS and domain name lookup are also prime vehicles for threat actors to launch cyberattacks, so DNS-based protections can act as an important early layer of defense against cyberattacks. 

Using DNS to advance cyberattacks is a serious threat, and Infoblox indicated 92% of malware uses DNS for command and control communication in a study published this year. To support the security choice of our customers, we’re partnering with Infoblox to deliver DNS Armor, a cloud-native DNS security service available now in preview. 

DNS Armor provides preemptive threat detection for internet-bound DNS queries initiated from Google Cloud workloads. It complements our existing cloud-first network security product portfolio by offering a foundational security layer that identifies DNS-based threats, including requests to malicious command and control (C2) servers, DNS tunneling for sensitive data exfiltration, and malware using DNS query. 

DNS Armor benefits from Infoblox’s preemptive DNS threat defense, which analyzes over 70 billion DNS events every day, and adds 4 million new threat indicators to their database every month. This preemptive approach to detection using DNS event analysis helps identify DNS-based threats 68 days earlier than complimentary security tools.

“As a customer of both Infoblox and Google Cloud, we’re excited about the transformative potential of this collaboration. Google Cloud’s DNS Armor, powered by Infoblox, is a leap forward in cybersecurity, combining the strengths of both companies to deliver visionary, proactive threat defense,” said Alfredo Rodriguez, vice president, Cloud Platform Infrastructure, Sabre Corporation.

Detecting threats early with DNS intelligence

DNS Armor provides both feed-based and algorithmic-based threat detection. 

• Feed-based: Detects known malicious, high-risk domains, and newly-registered domains likely to be used for malicious attacks.

• Algorithmic-based: Detects attacks using machine-learning based detection techniques, such as DNS tunneling attack techniques to prevent unauthorized data exfiltration.

DNS Armor also can help detect connections to malware distribution sites, and can pre-empt potential malicious downloads by blocking DNS queries going to malicious and high-risk domains. 

Many sophisticated cyberattacks establish a network connection with their command and control environment. You can use DNS Armor to get visibility into the earliest indicators of suspicious and malicious domains by detecting C2 activity, connections to malware distribution sites, and Domain Generation Algorithm (DGA) traffic originating from your workloads. 

For example, DNS Armor can detect evasive techniques like fast flux, which involves rapidly rotating the IP addresses linked to a single domain name, making detection of malicious domains more difficult and often missed by traditional security tools.

“The future of cybersecurity isn’t reactive — it’s preemptive,” said Mukesh Gupta, executive vice-president and chief product officer, Infoblox. “DNS sits at the foundation of every internet connection, making it the earliest — and most overlooked — opportunity to spot attacks before they begin. Google Cloud’s DNS Armor, integrated with Infoblox, detects threats weeks before they’re weaponized — helping security teams cut through noise and focus on what really matters — with a near-zero false positive rate. No more waiting for patient zero, no more blind spots. Just real-time threat visibility at scale.”

How DNS Armor works

It takes only a few clicks to start using DNS Armor for your Google Cloud workloads. Once active, DNS Armor intercepts internet-bound DNS queries generated from your Google Cloud workloads, and the queries are automatically inspected by the Infoblox Threat Defense service in real time. 

If a threat is detected, the Infoblox Threat defense service will generate a detailed threat log and store it in Cloud Logging. You can also send these threat logs to Security Command Center, Google Security Operations, or your preferred SIEM platforms for further analytics and incident response.

DNS Armor is easy to deploy and manage because it’s a turnkey managed service, with no virtual machines to manage and no impact to the performance of Cloud DNS. You can enable DNS Armor at the project-level across virtual PCs, giving you granular control over the cloud workloads that require protection.

Getting started

You can get started with DNS Armor by visiting our documentation page, and learn more about DNS security by checking out this video.

High-quality text embeddings are the engine for modern AI applications like semantic search, classification, and retrieval-augmented generation (RAG). But when it comes to picking a model to generate these embeddings, we know one size doesn’t fit all. Some use cases demand state-of-the-art quality, while others prioritize cost, speed, or compatibility with the open-source ecosystem.

To give you the power of choice, we’re excited to announce a major expansion of text embedding capabilities in BigQuery ML. In addition to the existing text-embedding-004/005 and text-multilingual-embedding-OO2 models, you can now use Google’s powerful Gemini embedding model and over 13K open-source (OSS) models directly within BigQuery. Generate embeddings with simple SQL commands, using the model that’s right for your job, right where your data lives.

Choosing an embedding model to fit your needs

With the addition of Gemini and a wide array of OSS models, BigQuery now offers a comprehensive list of text-embedding options. You can choose the right one that fits your needs, for model quality, cost, and scalability. Here’s a breakdown to help you decide.

Gemini embedding model in BigQuery ML

The Gemini embedding model is a state-of-the-art text embedding model that has consistently secured the top position on the Massive Text Embedding Benchmark (MTEB) leaderboard, a leading industry text-embedding benchmark. The model’s first-place ranking on the MTEB leaderboard directly demonstrates its high quality embedding. 

Here’s how to use the new Gemini embedding model in BigQuery:

1. Create a Gemini embedding model in BigQuery
Create the Gemini embedding model in BigQuery with the following SQL statement: by specifying the endpoint to be `gemini-embedding-001`:

2. Batch embedding generation
Now you can generate embeddings using the gemini-embedding model directly from BigQuery. The following example embeds the text in the bigquery-public-data.hacker_news.full dataset.

The Gemini embedding model utilizes a new quota control method known as Tokens Per Minute (TPM). For projects with a high reputation score, the default TPM value is 5 million. It’s important to note that the maximum TPM a customer can set without manual approval is 20 million.

The total number of rows a job can process depends on the number of tokens per row. For example, a job processing a table with 300 tokens per row can handle up to 12 million rows with a single job.

OSS embedding models in BigQuery ML

The OSS community is rapidly evolving the text-embedding model landscape, offering a wide spectrum of choices to fit any need. Offerings range from top-ranking models like the recent Qwen3-Embedding & EmbeddingGemma to small, efficient, and cost-effective small models such as multilingual-e5-small.

You can now use any Hugging Face text embedding models (13K+ options) deployed to Vertex AI Model Garden in BigQuery ML. To showcase this capability, in our example we will use multilingual-e5-small, which delivers respectable performance while being massively scalable and cost-effective, making it a strong fit for large-scale analytical tasks.

To use an open-source text-embedding model, follow these steps.  

1. Host the model on a Vertex endpoint
First, choose a text-embedding model from Hugging Face, in this case, the above-mentioned multilingual-e5-small. Then, navigate to Vertex AI Model Garden > Deploy from Hugging Face. Enter the model URL, and set the Endpoint access to “Public (Shared endpoint)”. You can also customize the endpoint name, region, and machine specs to fit your requirements. The default settings provision a single replica of the specified machine type below:

Alternatively, you can do this step programmatically using BigQuery Notebook (see the tutorial notebook example) or other methods mentioned in the public documentation.

2. Create a remote model in BigQuery
Deploying the model deployment takes several minutes. Once it’s complete, create the corresponding remote model in BigQuery with the following SQL statement:

Replace the placeholder endpoint in the above code sample with the endpoint URL. You can get information on endpoint_id from the console via Vertex AI > Online Prediction>Endpoints>Sample Request.

3. Batch embedding generation
Now you can generate embeddings using the OSS multilingual-e5-small model directly from BigQuery. See an example as follows, which embeds the text in the bigquery-public-data.hacker_news.full dataset.

By selecting the default resources for your model deployment, the query can process all 38 million rows in the hacker_news dataset in approximately 2 hours and 10 minutes — a baseline throughput achieved with just a single model replica. 

You can scale the workload by provisioning more computational resources or enable endpoint auto scaling. For example, if you choose 10 replicas, the query gets 10 times faster, and is able to handle billions of rows per a six-hour query job. 

4. Undeploy the model
Vertex AI endpoints accrue costs for as long as they are active, even when the machine is idle. To complete the batch job and prevent unexpected costs, you must “undeploy” the model after batch inference. You can do this through the Google Cloud console (Vertex AI > Endpoints > your endpoint > Undeploy model from endpoint), or by using our example Colab script.

For a batch workload, the most cost-effective pattern is to treat deployment, inference, and un-deployment as a continuous workflow, minimizing idle expenses. One way to achieve this is to run the sample Colab sequentially. The results are remarkable: processing all 38 million rows costs only about $2 to $3.

Get started today

Ready to build your next AI application? Start generating embeddings with Gemini or your favorite open-source models directly in BigQuery today. Dive into our documentation to learn the details, or follow our step-by-step Colab tutorial to see it in action.

We recently announced AI-first Colab Enterprise notebook experience in BigQuery and Vertex AI to help you simplify and transform your data science and analytics workflows. Colab Enterprise notebooks come with a built-in Data Science Agent to accelerate your data science development with agentic capabilities that facilitate data exploration, transformation, and machine learning modeling. With nothing but a simple prompt, the agent generates a detailed plan for your workflows – from data loading and cleaning to model training and evaluation.

Today, we’re introducing powerful new features in the Data Science Agent to further simplify and scale your analytical journeys, especially with large and open-format datasets.

Generate BigQuery ML, BigQuery DataFrames, & Spark

You can now harness the power of BigQuery Machine Learning (ML), BigQuery DataFrames (BigFrames), and Spark for large-scale data processing directly within the Data Science Agent. BigQuery ML and BigQuery DataFrames allow you to scale up data transformation, model training, and inference by running them directly on BigQuery. And with Serverless for Apache Spark, you can perform distributed data processing on large datasets, allowing you to work with data that is too large to fit into memory on a single machine.

To invoke these tools, simply include the following keywords in your prompt:

• For BigQuery ML: use “BigQuery ML”, “BQML”, or “SQL”
• For BigQuery DataFrames: specify “BigQuery DataFrames” or “BigFrames”
• For PySpark: include “Spark” or “PySpark”

In the future, the Data Science Agent will be able to pick the relevant framework for your use case — e.g., based on the size of your selected datasets or the contents of your Notebook.

In the meantime, here are some sample prompts to get you started:

• “Build a high-quality forecasting model using BigQuery SQL on project_id.dataset_id.table_id to predict stock needs. Present the model’s evaluation metrics and visualize the forecast with a 95% confidence interval.”

• “Using BigQuery DataFrames, train and evaluate a gradient boosted tree model to predict housing prices from the table project_id.dataset_id.table_id. Before training, one-hot encode the neighborhood column.”

• “I want to group similar customers together for targeted marketing campaigns, but first I need to do dimensionality reduction using a PCA model. Use Spark to do this on table project_id.dataset_id.table_id.”

Limitation: the Data Science Agent currently generates Spark 4.0 code. The agent can help you upgrade your code to Spark 4.0. However, if you need to use an earlier version of Spark, we recommend not using the Data Science Agent for PySpark for now.

Add data using context and @ mentions

We are also making it easier to bring your data into the conversation. The Data Science Agent can now automatically retrieve metadata and tables for your BigQuery tables. This means you can describe a table directly in your prompt and let the Data Science Agent search for the most relevant table on your behalf.

Further, you can now search for BigQuery tables within your current project using an @ mention. This familiar, industry-standard mechanism allows you to build your prompt with the relevant context — without your hands ever leaving the keyboard.

Limitation: The @ mention currently only searches for BigQuery tables in your current project. For broader searches across projects or to add files from session storage and local uploads, please continue to use the “+” button.

Try the Data Science Agent today

Under the hood, we’ve also optimized the Data Science Agent so it will start up faster after your first message. Less waiting, faster insights. Similar improvements for Colab Enterprise in Vertex AI are coming soon.

We’re committed to evolving the AI-powered data science experience and can’t wait to show you what we’re building next. To get started, check out the resources below:

• Access:

• BigQuery: Navigate to Google Cloud Console > BigQuery > Notebook 

• Vertex AI: Navigate to Google Cloud Console > Vertex AI > Colab Enterprise (Note: BigQuery ML, BigQuery Dataframes, and Spark improvements mentioned here are not yet available in Vertex AI – but are coming soon.)

• Use the Data Science Agent with BigQuery

• Use the Data Science Agent with Vertex AI

Today, Google announced the Agent Payments Protocol (AP2), an open protocol developed with leading payments and technology companies to securely initiate and transact agent-led payments across platforms. The protocol can be used as an extension of the Agent2Agent (A2A) protocol and Model Context Protocol (MCP). In concert with industry rules and standards, it establishes a payment-agnostic framework for users, merchants, and payments providers to transact with confidence across all types of payment methods.

We’re collaborating with a diverse group of more than 60 organizations to help shape the future of agentic payments, including Adyen, American Express, Ant International, Coinbase, Etsy, Forter, Intuit, JCB, Mastercard, Mysten Labs, Paypal, Revolut, Salesforce, ServiceNow, UnionPay International, Worldpay, and more.

Why is a protocol needed?

AI agents are capable of transacting on behalf of users, which creates a need to establish a common foundation to securely authenticate, validate, and convey an agent’s authority to transact. While today’s payment systems generally assume a human is directly clicking “buy” on a trusted surface, the rise of autonomous agents and their ability to initiate a payment breaks this fundamental assumption and raises critical questions that AP2 helps to address, including:

• Authorization: Proving that a user gave an agent the specific authority to make a particular purchase.

• Authenticity: Enabling a merchant to be sure that an agent’s request accurately reflects the user’s true intent. 

• Accountability: Determining accountability if a fraudulent or incorrect transaction occurs. 

AP2 is an open, shared protocol that provides a common language for secure, compliant transactions between agents and merchants, helping to prevent a fragmented ecosystem. It also supports different payment types–from credit and debit cards to stablecoins and real-time bank transfers. This helps ensure a consistent, secure, and scalable experience for users and merchants, while also providing financial institutions with the clarity they need to effectively manage risk.

How it works: Establishing trust via mandates and verifiable credentials

AP2 builds trust by using Mandates—tamper-proof, cryptographically-signed digital contracts that serve as verifiable proof of a user’s instructions. These mandates are signed by verifiable credentials (VCs) and act as the foundational evidence for every transaction.

Mandates address the two primary ways a user will shop with an agent:

• Real-time purchases (human present): When you ask an agent, “Find me new white running shoes,” your request is captured in an initial Intent Mandate. This provides the auditable context for the entire interaction in a transaction process. After the agent presents a cart with the shoes you want, your approval signs a Cart Mandate. This is a critical step that creates a secure, unchangeable record of the exact items and price, ensuring what you see is what you pay for.

• Delegated tasks (human not present): When you delegate a task like, “Buy concert tickets the moment they go on sale,” you sign a detailed Intent Mandate upfront. This mandate specifies the rules of engagement—price limits, timing, and other conditions. It serves as verifiable, pre-authorized proof that can allow the agent to automatically generate a Cart Mandate on your behalf once your precise conditions are met.

In both scenarios, this chain of evidence culminates in securely linking your payment method to the verified contents of the Cart Mandate. This complete sequence—from intent, to cart, to payment—creates a non-repudiable audit trail that answers the critical questions of authorization and authenticity, providing a clear foundation for accountability.

Unlocking new commerce experiences

AP2’s flexible design provides a foundation to support both simple and entirely new commercial models. Let’s consider a few examples below, which all assume Intent Mandates have been signed on behalf of a user: 

• Smarter shopping: A customer discovers a winter jacket they want is unavailable in a specific color, so they tell their agent: “I really want this jacket in green, and I’m willing to pay up to 20% more for it.” The agent then monitors prices and availability and automatically executes a secure purchase the moment that specific variant is found, capturing a high-intent sale that would have otherwise been lost.

• Personalized offers: A shopper tells their agent they want a new bicycle for an upcoming trip from a specific merchant. Their agent communicates this information—which includes the trip’s date—to the merchant, whose own agent can respond by creating a custom, time-sensitive bundle offer that includes the bike, a helmet, and a travel rack at a 15% discount, turning a simple query into a more valuable sale.

• Coordinated tasks: A user is planning a weekend trip and tells their agent: “Book me a round-trip flight and a hotel in Palm Springs for the first weekend of November, with a total budget of $700.” The agent can then interact with both airline and hotel agents, as well as online travel agencies and booking platforms, and once it finds a combination that fits the budget, it can execute both cryptographically-signed bookings simultaneously.

Support for emerging payments systems

AP2 is designed as a universal protocol, providing security and trust for a variety of payments like stablecoins and cryptocurrencies. To accelerate support for the web3 ecosystem, in collaboration with Coinbase, Ethereum Foundation, MetaMask and other leading organizations, we have extended the core constructs of AP2 and launched the A2A x402 extension, a production-ready solution for agent-based crypto payments. Extensions like these will help shape the evolution of cryptocurrency integrations within the core AP2 protocol. 

What’s next: A call for collaboration 

AP2 provides a trusted foundation to fuel a new era of AI-driven commerce. It establishes the core building blocks for secure transactions, creating clear opportunities for the industry–including networks, issuers, merchants, technology providers, and end users–to innovate on adjacent areas like seamless agent authorization and decentralized identity. We are committed to evolving this protocol in an open, collaborative process, including through standards bodies, and  invite the entire payments and technology community to build this future with us. 

Many of the partners building A2A agents have extended their support to AP2. This growing ecosystem will continue to make their agents available in our AI Agent Marketplace, including new, transactable experiences enabled by AP2. For example, enterprise companies could use AP2 for B2B applications, such as enabling autonomous procurement of partner-built solutions via Google Cloud Marketplace or the automatic scaling of software licenses based upon real-time needs.

To get started, visit our public GitHub repository to review the complete technical specification, documentation, and reference implementations. Moving forward, this repository will be updated regularly with additional reference implementations from Google and innovations from the community to demonstrate the power and scalability of AP2.

Support from our ecosystem

• Accenture: “Google Cloud’s Agent Payments Protocol (AP2) complements the Agent2Agent protocol and Model Context Protocol to provide a unified framework for agents to transact,” “Innovations like this will enable many of the agentic solutions that reinvent payments for clients – not only for today’s needs, but for the evolving models of future commerce.” – Scott Alfieri, Google Business lead at Accenture
• Adobe: “Adobe is proud to work with Google to advance secure and authenticated agentic commerce – our role in the Agent Payments Protocol (AP2) underscores our commitment to trusted, AI-driven experiences. With Adobe Commerce and AI agents powering customer journeys, we are focused on delivering secure, reliable, and authentic transactions for businesses and consumers.”  – Loni Stark, VP of Strategy and Product at Adobe
• Adyen: “Agentic commerce is not just about a consumer-facing chatbot, but about the underlying infrastructure that powers it all. Adyen’s collaboration on Google’s Agent Payments Protocol (AP2) is a natural extension of our mission to provide the merchants with the payments building blocks for tomorrow’s commerce. We’re excited to help establish a common rulebook that ensures security and interoperability for everyone involved in the payments ecosystem.” – Ingo Uytdehaage, Co-CEO at Adyen
• Airwallex: “Airwallex is thrilled to support Google’s Agent Payments Protocol (AP2). This is a critical step forward in building a secure, interoperable ecosystem for agentic AI payments. This protocol gives businesses and consumers the confidence to delegate tasks to AI agents, aligning with our mission to build the future of finance by empowering businesses globally.” – Jacob Dai, Co-Founder & CTO at Airwallex
• American Express: “With the rise of AI-driven commerce, trust and accountability are more important than ever. American Express is excited to contribute to the creation of AP2 as a protocol intended to protect customers and enable participation in the next generation of digital payments.” – Luke Gebb, EVP, Amex Digital Labs, American Express
• Ant International: “Ant International is excited to partner with Google on protocol-setting for practical AI applications in agentic commerce to unlock new merchant growth and elevate consumer experience, by leveraging our expertise in alternative payment methods and trusted AI innovations.” – Jiangming Yang, Chief Innovation Officer at Ant International
• BHN: “As a trusted partner processing billions of transactions globally, BHN is excited to help shape emerging protocols like AP2 that will enable both merchants and consumers to leverage the power of stored value in secure, autonomous commerce, enabled by AI Agents.” – Nik Sathe, CPTO at BHN
• BVNK: “Stablecoins provide an obvious solution to the scaling challenges agentic systems are already facing with legacy financial infrastructure. We at BVNK were extremely excited to hear that Google has been working on solving this problem and couldn’t wait to contribute” – Donald Jackson, CTO at BVNK
• Checkout.com: “Agentic commerce is reshaping the checkout moment, and Google’s Agent Payments Protocol (AP2) is a pivotal step forward. At Checkout.com, we’re proud to support open protocols that strengthen trust and give merchants the flexibility to meet their customers where they are, however they want to shop.” – Meron Colbeci, Chief Product Officer, Checkout.com
• Coinbase: “x402 and AP2 show that agent-to-agent payments aren’t just an experiment anymore, they’re becoming part of how developers actually build. Bringing x402 into AP2 to power stablecoin payments made sense – it’s a natural playground for agents to start transacting with each other and testing out crypto rails. And it’s exciting to see the idea of agents paying each other resonate with the broader AI community.” – Erik Reppel, Head of Engineering at Coinbase Developer Platform
• Crossmint: “With Crossmint’s tools, developers can let agents buy anything using both credit cards or stablecoins. Our goal is to unlock instantaneous, global commerce, giving agent builders the greatest flexibility. Our partnership with Google on AP2 represents our commitment that agentic commerce wins everyone’s trust as a secure, reliable, and seamless way to transact. Time to accelerate!” – Alfonso Gomez, Co-founder at Crossmint
• Confluent: “Confluent is excited to support Google in this effort to build an open, secure, and high-trust payments protocol. Agent Payments Protocol (AP2) aligns perfectly with our vision of a real-time data-driven world, and we believe our expertise in data streaming with Apache Kafka will be critical in creating a resilient and scalable payments ecosystem for the agentic web.” – Pascal Vantrepote, Partner CTO at Confluent 
• Dell: “At Dell Technologies, we’re dedicated to making agentic AI a reality for businesses worldwide. The transformative potential of agentic automation hinges on trust, security and standardization, especially for customer-facing eCommerce platforms. By supporting the Agent Payments Protocol (AP2) with Google, we’re laying the groundwork for a future where AI-driven commerce is reliable, accessible, and trusted by all.” – Satish Iyer, Vice President, Innovation & Ecosystems, Office of the CTO, Dell Technologies
• Deloitte: “As Agentic Commerce rapidly emerges as a transformative force, the industry will need robust standards to empower AI agents to transact payments securely and effectively. These standards must address critical areas such as security, identity, frictionless commerce, trust, and privacy, all while providing compatibility with the existing global payments infrastructure. Deloitte is proud to help shape this evolving industry alongside Google, extending the widely adopted A2A protocol to enable agent-driven payments and commerce.” – Gopal Srinivasan, Alphabet Google Alliance Global AI & Data Leader at Deloitte Consulting LLP
• DLocal: “Payment agents are no longer an idea, they’re rapidly becoming a reality. In the dynamic emerging markets we serve, payments are fragmented and complex, from cards to local payment methods, to wallets and stablecoin.Agent Payments Protocol (AP2) turns that complexity into a single, interoperable framework, enabling agent-initiated payments that are safe, seamless, and designed to boost merchant conversion while keeping users in control.” – Pedro Arnt, CEO at DLocal
• Ebanx: “Agent Payments Protocol  (AP2) will power the next era of commerce, and to build a safe and secure environment for this is now the most important step. EBANX is proud to be part of this effort with Google.” – Eduardo de Abreu, Vice President of Product at Ebanx
• Eigen Labs: “Google’s new Agent Payments Protocol (AP2) is a major step toward a future where AI agents are meaningful economic actors, whether that’s on behalf of humans, organizations or themselves. EigenCloud is proud to partner with Google on this initiative to provide the verifiability infrastructure that ensures these agents are held accountable by any counterparty. Together, we’re helping create a global verifiable economy where agents can coordinate, transact, and prove their actions to humans and to each other.” – Sreeram Kannan, Founder & CEO at Eigen Labs 
• Fiuu: “”As agentic commerce reshapes payments infrastructure, Fiuu supports open protocols like A2A and AP2 to enable secure, scalable agent-to-agent transactions across multi-channel systems, advancing interoperability, trust, and inclusive payment ecosystems.” – Eng Sheng Guan, CEO at Fiuu
• Forter: “At Forter we believe in the potential of agents to revolutionize commerce and we are proud to collaborate with Google in creating modern protocols that benefit brands, consumers and AI developers alongside Forter’s Trusted Agentic Commerce Protocol (TACP).” – Michael Reitblat, CEO at Forter
• Gr4vy: “We are proud to support this new open protocol (AP2). By working together as an industry, we can ensure this next chapter of payments is built on trust, transparency and flexibility.” – John Lunn, CEO and Founder at Gr4vy
• Gravitee: “In the agentic world, secure and trusted transactions demand open protocols. Google’s Open Standard for Agent Payments Protocol (AP2) addresses this need. Gravitee’s Agent Mesh already supports A2A and MCP with a strong focus on security and governance, and we are committed to extending this support so customers in financial services, retail, and beyond can confidently benefit” – Linus Hakansson, Chief Product Officer at Gravitee 
• Global Fashion Group: “Integrating A2A and MCP into the Agent Payments Protocol (AP2) enables a modular, interoperable architecture with versioned contracts, making integration and testing straightforward. Modern payments, engineered for scale—secure, seamless, and built to power global commerce.” – Quy Tran, Director of Engineering at Global Fashion Group
• Intuit: “Intuit focuses on enabling the financial success of consumers, businesses, and accountants. We are excited to leverage our AI and data capabilities to help develop the open Agent Payments Protocol (AP2) to create better experiences for all. Our technologists will have the ability to use the protocol to deploy AI agents towards autonomous financial workflows as part of our done-for-you experiences for customers.” – Tapasvi Moturu, Vice President, Software Engineering at Intuit
• JCB: “JCB champions Google’s Agent Payments Protocol (AP2) initiative as the innovative and important protocol that will unlock a new era of payments, and JCB looks forward to contributing to the protocol to benefit our entire ecosystem, including our banking and payment institution partners, cardmembers, and merchants.” – Shinya Kubotera, Executive Officer & Head of Strategic Innovations at JCB co., Ltd
• JusPay: “The future of payments is inherently open and interoperable – our work with UPI and the development of Hyperswitch, the world’s first open-source payments orchestration platform has demonstrated this power. We believe this new protocol (AP2) provides the secure, shared foundation needed to make AI-driven commerce a reality, and we are ready to contribute our expertise to this initiative.”  – Sheetal Lalwani, Co-founder at Juspay
• KCP: “NHN KCP endorses the Agent Payments Protocol (AP2) as a key advancement in the global payments ecosystem and looks forward to collaborating with global partners to help make AI-based payments more reliable, convenient, and widely adopted.” – Jae-wook Noh, Executive Managing Director at NHN KCP  
• Lightspark: “Having worked on payments at Google, I’ve seen how open and verified protocols can unlock powerful network effects. Google’s Agent Payments Protocol (AP2) is a big step toward a future where trusted AI agents transact seamlessly on our behalf. At Lightspark, we’re committed to that vision of open, global interoperability.” – Alberto Martin CPO at Lightspark
• ManusAI: “Google’s Agent Payments Protocol (AP2) represents a breakthrough solution that finally addresses the fundamental monetization challenges we’ve long faced in the agent ecosystem—enabling seamless, standardized compensation between AI agents while eliminating the unsustainable resource imbalances that have hindered true multi-agent collaboration.” – Tao Zhang, CTO at Manus
• Mastercard: “Mastercard is committed to ongoing, responsible innovation – and we are excited to be collaborating with Google, leading banks, merchants, AI platforms and other industry leaders to help shape the future of agentic commerce. These efforts include critical work with standards bodies such as the FIDO Alliance, where we are advancing verifiable credentials to capture and secure consumers’ intent in this dynamic new context. Together, we’re playing an essential role in securing the payments ecosystem – ensuring that trust and safety remain at the core of every transaction.” – Pablo Fourez, Chief Digital Officer at Mastercard 
• MetaMask: “Blockchains are the natural payment layer for agents, and Ethereum will be the backbone of this. With Agent Payments Protocol (AP2) and x402, MetaMask will deliver maximum interoperability for developers and will enable users to pay agents with full composability and choice—while retaining the security and control of true self-custody” – Marco De Rossi, AI Lead at MetaMask
• Mesh: “For AI to truly drive commerce, agents need a secure and universal way to handle payments. Google’s new Agent Payments Protocol (AP2) is a huge step forward, providing the foundational framework to make this possible. We’re proud to support this effort because it unlocks the full potential of agent-led commerce, particularly with programmable assets like crypto. Our technology abstracts away the complexity of the crypto ecosystem, giving agents seamless access to hundreds of wallets and exchanges and supporting over 100 tokens. This ensures payments are not just completed, but are routed through the most efficient paths to guarantee speed and success.” – Bam Azizi, CEO and co-Founder at Mesh
• Mysten Labs: “Verified agents making purchases on behalf of verified users is the next frontier for AI-powered automation.  Google’s Agent Payments Protocol (AP2) combines programmable payments via modern blockchains like Sui with open protocols like A2A and MCP that are enjoying rapid growth. It’s the perfect substrate for real-world agentic commerce.” – Sam Blackshear, Chief Technology Officer and Co-Founder at Mysten Labs, the original contributor to Sui.
• Nexi: “We are delighted to partner with Google Cloud on AP2 in order to contribute to shape a fundamental paradigm shift in commerce. As part of our DNA of being European by scale and local by nature, we aim at empowering European merchants to continue to compete on a global scale, while delivering frictionless and personalised online shopping experiences to consumers. Leveraging Agentic AI eCommerce technology from Google Cloud, we will continue to simplify payments for our merchants and partners” – Roberto Catanzaro, Chief Business Officer Merchant Solutions at Nexi
• Okta: “Extending the A2A protocol into payments is an important step toward building a secure, interoperable foundation for commerce between AI agents. At Auth0, we’re excited to support the Agent Payments Protocol (AP2) and help ensure that future payments between AI agents are both seamless and secure.” – Stephen Lee, Vice President, Technical Strategy and Partnerships at Okta
• Payoneer: “At Payoneer, we see enormous potential in AI agents to simplify financial workflows for millions of small businesses. By supporting the Agent Payments Protocol (AP2), we’re ensuring agents can collaborate securely and seamlessly, just as our platform connects SMBs worldwide.” – Guy Shalev, Vice President of AI at Payoneer
• PayPal: “AP2 provides the critical foundation for trusted agent payments, giving the ecosystem much needed clarity on how to facilitate trusted transactions. PayPal is fully aligned with this vision and excited to build on it, bringing our commerce expertise to help extend these principles across the entire purchase journey.” – Prakhar Mehrotra, SVP and Global Head of AI at PayPal
• PwC: “PwC is committed to fostering innovation with agentic AI that focuses on maintaining trust, safety and privacy for critical tasks like payments and money movement broadly.  We believe the Agent Payments Protocol (AP2) and extension to the Agent2Agent protocol represent a significant leap forward, enhancing safety without compromising information.” – Scott Likens, Global / US Chief AI Engineering Officer at PwC
• Salesforce: “With extensive expertise in powering digital commerce, Salesforce is excited to help businesses harness agentic payments at scale – creating truly frictionless commerce experiences and driving the productivity that is crucial to becoming an Agentic Enterprise today. ” – Nitin Mangtani, SVP & GM Commerce & Retail Cloud at Salesforce 
• Shopee: “At Shopee, we see immense potential for agents to transform e-commerce, and believe that industry protocols such as Google’s Agent Payments Protocol (AP2) will be critical to enabling this future.” – David Chen, Chief Product Officer at Shopee 
• Worldpay: “Worldpay shares Google’s vision of an open, interoperable foundation for agentic commerce, built on trust and safety to empower merchants and shoppers. The AP2 protocol represents a meaningful first step in defining how agents, merchants, and payment providers can transact securely, at scale.” – Cindy Turner, Chief Product Officer at Worldpay
• 1password: “Open protocols like A2A and AP2 are critical to driving broad adoption of AI while ensuring security and transparency remain foundational. At 1Password, we see support for digital payment credentials as just the beginning. The future is multi-agent, and managing agent access and authorization starts with securing credentials, all while upholding our core security values of privacy, transparency, and trust.” – Anand Srinivas, Vice President, Product & AI at 1Password

Get started by checking out our public GitHub repository to see the complete technical specification, documentation, and reference implementations.

Data science is undergoing a profound transformation with new tools and platforms empowering practitioners like never before. To help you navigate this evolving landscape, we’re excited to announce the release of our new ebook: A Practical Guide to Data Science with Google Cloud.

Google Cloud provides extensive data science-related tooling, and this new practical guide builds on these offerings:

We’ve designed our new guide for practitioners looking to use Google Cloud’s capabilities across BigQuery, Vertex AI, and Google Cloud Serverless for Apache Spark. We’ll show you how to use an AI-first approach in your data science workflows, use previously untapped unstructured and multimodal data, and achieve new levels of efficiency.

Let’s take a look at what’s inside!

Unified, streamlined workflows

The first section of the ebook explores how Google Cloud helps reduce friction in the data science workflow. We highlight the shift toward a central workspace that blends SQL, Python, and Spark code in the same notebook. Tools like Colab Enterprise notebooks are integrated with assistive features that help you generate multi-step plans and code from high-level goals, allowing you to focus on analysis and interpretation.

End-to-end tooling on a unified data foundation

The ebook also details how a unified data foundation is key to modern data science. With Google Cloud, you can manage structured and unstructured data in one place, and use familiar SQL syntax to process documents or analyze images directly. The guide walks through a spectrum of model development options on the platform – whether you prefer SQL with BigQuery Machine Learning, the flexibility of Spark, a pandas and scikit-learn syntax with BigQuery DataFrames, or custom model development with Vertex AI.

Real-world use cases and code

A significant portion of this ebook is dedicated to common, real-world data science use cases we typically see customers face. These examples show how to tackle projects like modernizing retail demand forecasts or assessing environmental risks for agricultural investments. Each use case comes with a link to a notebook, so you can get started right away.

Download the ebook today!

We hope this ebook will serve as a useful, hands-on resource to improve your data science workflows.

Download your free copy: A Practical Guide to Data Science with Google Cloud.

Today, we are thrilled to announce the 14 innovative startups selected to join the Google for Startups Accelerator: AI First in the Middle East, North Africa and Turkey.

These companies are leveraging AI to tackle significant challenges and create new opportunities across various industries. From real estate and finance to enhancing healthcare and industrial safety, this cohort represents the forefront of technological advancement in the region.

The Accelerator is designed to support these startups in their journey to scale, providing comprehensive mentorship, technical training, and resources focused on product development, business strategy, and responsible tech.

Please join us in welcoming the startups participating in the Google for Startups Accelerator: AI First program:

Abwab.ai: An AI-powered credit assessment platform that empowers financial institutions to make faster, fairer, and more accurate lending decisions for SMEs.

COGNNA: An agentic SecOps platform that leverages AI agents to autonomously investigate, analyze, and respond to security threats at machine speed.

Distichain: A TradeTech platform automating import and export, connecting all participants of a trade to an ecosystem of verification, payments, and supply-chain service providers.

Eluvium: Builds AI agents to automate sales operations & procurement for manufacturers.

FoxPrep: Brings Duolingo-style exam prep to local high-stakes exams where students currently have no quality study options.

Heemera: A global provider of crypto investigative, blockchain analytics, and assets recovery services.

iWarranty: A unified platform to transform warranty management.

Malakah: A digital platform based on AI that provides legal services in an easy and reliable manner, without any human intervention.

Monit: A pioneering provider of modular digital solutions for industrial safety, efficiency, and asset management in emerging markets.

Navatech: An AI-first, cloud-native platform empowering frontline workers starting with health and safety in construction and other high-risk industries.

Re-skill: An AI-native learning infrastructure that transforms ideas and internal documents into engaging videos, quizzes, and learning paths.

Sponix Tech: An award-winning sports technology startup that has developed unique solutions for leagues, clubs, and broadcasters.

VenueX: An AI-powered omnichannel performance platform that connects in-store data with digital marketing channels to optimize offline ROAS, boost store visits, and improve local visibility.

xBites: The MENA region’s first AI-powered real estate sales assistant, built to qualify and convert leads instantly over WhatsApp.

Please join us in welcoming this cohort to the Google for Startups Accelerator: AI First program. Visit the program page to learn more and sign up for future cohorts from the MENA and Turkey region.

Are you spending more time debugging ML model performance than you are building? You’re not alone. In today’s fast-paced AI landscape, optimizing models is a complex challenge, from navigating new model architectures to dealing with the ever-changing hardware and software stacks. Even at Google, where we optimize models for products like Gemini, Search, and YouTube, we’ve faced these exact challenges.

That’s why we’re providing a key tool to the community from our internal toolbox: XProf, our core ML profiler. In this post, we’ll show you how to use this powerful tool and the new Cloud Diagnostics XProf library to easily identify model bottlenecks, find hidden opportunities for optimization, and get back to what you do best: innovating.

Updated XProf

A few years ago, we released a fork of our XProf tool as open source called “TensorFlow profiler”, accessible through TensorBoard as “tensorboard_plugin_profile”, which many ML Engineers (MLEs) in the community were using to optimize ML models on xPUs. However, given the rapid changes in the market and the huge advancements made to our Google internal XProf tool, we decided it was time to share these advanced features with the community. We have recently merged the functionality of internal XProf to our external tool, moved the external code base under OpenXLA, and renamed our external profiler to simply “XProf”, reflecting its evolution beyond TensorFlow and TensorBoard. Now, the community can enjoy the benefits of using the same profiling tool that is being used within Google and Google DeepMind. Another major benefit of moving XProf under OpenXLA is that we will now support all XLA based frameworks equally, including JAX, Pytorch/XLA and TensorFlow/Keras, with the same consistent experience.

Capture profiles

In order to use XProf, you need to first enable profile capture within your model workload code. There are two ways to capture profiles:

• Programmatic capture– annotate your model code in order to specify where in your code you want to capture profiles.
• On-demand capture (also known as, manual capture)– for time periods when you did not enable programmatic profile capture, you can still capture profiles ad-hoc during your run. You can use this if you see a problem with your model metrics during the run and want to capture profiles at that point for some period in order to diagnose the problem. 

Visualize profiles

In order to visualize your captured profiles, we recommend you use TensorBoard, either locally or on Google Cloud. In this section, we will focus on how we have enabled a seamless self-hosted profiling experience on Google Cloud with XProf and TensorBoard with a library called Cloud Diagnostics XProf (also known as, XProfiler library) which is available in open source on GitHub.

With this library, we make the following improvements over running XProf and TensorBoard locally:

• Easy setup and packaging of XProf and TensorBoard dependencies
• Store your profiles in Google Cloud Storage which can be useful for long term retention and post-run analysis (local profiles captured will be deleted after researcher finishes run)
• Fast loading of large profiles and multiple profiles by provisioning TensorBoard on Google Compute Engine VM or Google Kubernetes Engine pod, with the option to change machine type based on user needs for loading speed and cost. 
• Create a link for easy sharing of profiles and collaboration with team members and Google engineers
• Easier on-demand profiling of workloads on GKE and Compute Engine instances to choose any host running your workload to capture profiles 

Set up the Cloud Diagnostics XProf library

First, capture the profiles in a storage path similar to gs://<some_bucket>/<some_run>/

Next, set up the gCloud CLI and a Python virtual environment.

Finally, install the library with the following command: 

pip install cloud-diagnostics-XProf

Create TensorBoard instance with XProfiler

Create a VM or GKE pod to host TensorBoard: 

XProfiler create -z $ZONE -l $GCS_PATH

or

XProfiler create --GKE -z $ZONE -l $GCS_PATH

Visualize Profiles on TensorBoard

One of the major changes we made to the TensorBoard profile plugin recently is that we relaxed the profile path requirements so that every TensorBoard instance can load profiles from multiple runs in a directory. This means you can now easily compare profiles across runs or between sessions in a long-running job without having to launch separate TensorBoard instances, speeding up your analysis workflow.  During XProfiler create, you can point the TensorBoard instance to the root storage path gs://<some_bucket> and this will allow TensorBoard to pick up all profiles under this root directory.  You can see in the TensorBoard UI below how you will see the profiles for all the runs and sessions as run1/session1, run1/session2, run2/session1 in the drop down.

Faster profile loading and tool transitions

Another major change we have made to the TensorBoard XProf plugin is improving loading time. We added support for multithreading to the plugin, which allows larger profiles to load much faster. We also added caching, which allows the second load to be even faster, and allows users to move smoothly between the tools in XProf while doing their performance optimization.

Machine options for hosting TensorBoard + XProf with Xprofiler

By default, XProfiler create will create a VM, specifically c4-highmem-8 VM. You can change the machine type with the -m flag. Also, if you want to create a TensorBoard instance on a GKE pod, you can pass —GKE flag to XProfiler create. Some customers prefer to have their TensorBoard instance hosted on a GKE pod as it makes it easier to manage this TensorBoard instance along with the rest of their workload deployed on GKE.

The VM or GKE pod that hosts TensorBoard makes loading of large profiles and multiple profiles much faster on Google Cloud than locally hosted TensorBoard. Based on our benchmarking, profiles of order of 1GB load will load within a few minutes for the first load using the default c4-highmem-8 VM. You can choose different machine types based on your performance and cost needs. 

Link for sharing profiles

After you run XProfiler create, you will get something like the following message

Instance for gs://<some-bucket> has been created.

You can access it via following,

1.https://<id>-dot-us-<region>.notebooks.googleusercontent.com.

2. XProfiler connect -z <some zone> -l gs://<some-bucket> -m ssh

Instance is hosted at XProf-97db0ee6-93f6-46d4-b4c4-6d024b34a99f VM.

Note: The first option (1) is a link that has been created which you can just click and view your XProf profiles on TensorBoard. Performance optimization is a very iterative and collaborative process, so in order to enable this collaboration, the Cloud Diagnostics XProf library creates a link to  he TensorBoard instance so that users can easily share their profiles with their teams and with Google engineers helping with performance optimization on Google Cloud. You control who has access to the link based on permissions set for the Cloud Storage bucket that the TensorBoard instance is pointing to. 

In case the link doesn’t work for some reason, we also provide a way to SSH into the TensorBoard instance in order to view your profiles using XProfiler connect command.

On-demand profile capture

If you enabled the profiler server in your workload code and want to perform on-demand profiling, you can do this in two ways:

1. Click on the “capture profile” button on TensorBoard UI. We support on demand capture for workloads running on GKE and Compute Engine.

2. Use XProfiler capture in CLI, providing similar information as your would through the “Capture profile” button on the TensorBoard UI.

New capabilities of XProf

With the updated XProf, users will see some many updated features for the most popular tools which include:

• Trace viewer
• Memory viewer/memory profile
• Graph Viewer
• HLO Op profile/HLO Op stats
• Overview page

Most notably, on the memory viewer, you can now see 7 different types of memory including HBM (high bandwidth memory), host, and for TPUs – SparseCore, VMEM, SMEM, CMEM and Sync Flags (SFlag).

You will also see many of the links from trace viewer and HLO Op profile back to Graph viewer work seamlessly for all Ops. We have also improved source line visibility to cover more Ops. 

The most common flow used for finding performance bottlenecks using XProf looks something like the following:

1. MLE opens up XProf in TensorBoard and looks at different ops in trace viewer or HLO op profile

2. They click on the op that they interested in digging into to get more details

3. For this op, they click on the link to graph viewer to see how the Op is placed in their model

4. They take a look at memory viewer to see if HBM/host/SparseCore memory is utilized efficiently for the model and for the specific Op

5. Once they have determined which Ops they want to optimize, they look at the source code line for those Ops in order to implement any optimizations. 

The updated XProf tool makes this entire flow smooth and easy.  

New XProf tools

In addition, we also have released a few new tools in XProf including: 

• Framework op stats– performance statistics of framework-level operations (e.g., JAX or TensorFlow).
• Roofline– visually see whether your program is memory-bound or compute-bound, and how close the program’s performance is to the hardware’s theoretical peak performance, represented as a “roofline”.
• Megascale stats–  analyze multi-slice communication performance of workloads spanning multiple TPU slices that communicate across the Data Center Network (DCN).
• GPU kernel stats–  performance statistics and the originating framework operation for every GPU-accelerated kernel that was launched during a profiling session.

Pallas Kernel Visibility in XProf

One of the big areas of performance visibility that has been requested in XProf is around visibility of performance for Pallas kernels. These kernels were displayed in XProf as “custom calls”, but it was hard to see details of the performance of the custom call and its implementation. We are very happy to announce increased support and visibility for Pallas kernels within XProf for you. Now, you can see more details of your Pallas kernel in both HLO Op Profile as well as Graph Viewer. For each Pallas kernel custom call, you will see the name of the kernel if it is a common Pallas kernel, and when you click it, you will see performance and other information about the kernel in the side panel. To get accurate performance metrics when clicking on the kernel in the side panel, the kernel author must provide a cost model by passing a pl.CostEstimate object to their pallas_call function. In addition, there is a “custom call text” button where the user can see more details about the Pallas kernel implementation.

CUDA Graph Tracing with XProf

XProf also supports profiling on NVIDIA GPUs for workloads using XLA. Most of the XProf tools support NVIDIA GPUs, but the view of GPU traces is different as GPU traces are organized by CUDA streams. You capture profiles for JAX on GPUs just as previously mentioned, and most XProf tools support NVIDIA GPU performance views. One new feature we have added support in XProf for is CUDA Graphs. CUDA Graphs allow users to optimize kernel launch overheads by allowing users to define larger units of work as graphs that combine individual ops. XProf now allows users that use CUDA Graphs to now trace those with XProf for further performance optimization. 

To enable CUDA Graph tracing, you need to enable the relevant XLA flag <XLA_FLAGS=--xla_enable_command_buffers_during_profiling>.

This will allow you to see the CUDA Graph in your XProf trace. You can also see the CUDA Graph in Graph Viewer as well. 

To see the traces of the different nodes broken down within the CUDA Graph, you can add an advanced config value for ProfilerOptions <gpu_enable_cupti_activity_graph_trace , True>. 

You will now see the detailed nodes traced within the graph, as well as arrows linking hosts to the specific stream where the graph node execution commenced.

What customers are saying

There are many TPU and GPU customers on various frameworks that have benefited from the updated XProf tool and Cloud Diagnostics XProf library. Here are a couple of quotes to give you a sense of what they see.

“We are a technology hub, building next-gen apps using world-class expertise and cutting-edge tech.  The improved XProf tool has helped us optimize our models and be more productive on both JAX and PyTorch XLA on TPUs. XProf lets us drill down from an identified bottleneck in the trace viewer to the graph view of our model to the source code so we know what we need to modify.  We can now focus on tuning our code for the best performance with TPUs.  With the Cloud Diagnostics XProf library, it is very easy to capture and load profiles for long term analysis and sharing links makes collaboration with others a breeze!”

– Mustafa Ozuysal, Senior ML Researcher, HubX

“Mathpix is on the cutting edge of computer vision, changing the way that individuals and businesses interact with documents.  For our image inference performance challenge, XProf helped us collect and analyze profile traces through which  we discovered our cache usage was the problem. Tuning this code improved our performance with JAX on TPUs.  The Cloud Diagnostics XProf library made it easy for me to set up XProf on Google Cloud and load my profiles super fast!”

– Remy Ochei, Senior Machine Learning Engineer, Mathpix

Conclusion 

Whether you’re looking to identify bottlenecks to overlap compute and communication, optimize memory usage on-device, or get deep insights into custom kernel performance, the updated XProf and the new Cloud Diagnostics XProf library provide a comprehensive, end-to-end, and collaborative profiling solution. We’ve brought Google’s internal tools to the entire community to help you achieve peak performance for your models on Google Cloud.

Get started today by checking out the following resources:

• XProf tools documentation: https://openxla.org/XProf
• XProf github: https://github.com/openxla/XProf
• Cloud-diagnostics-XProf library github: https://github.com/AI-Hypercomputer/cloud-diagnostics-XProf
• Profiling on Google Cloud: https://cloud.google.com/tpu/docs/profile-tpu-vm

In addition, please check out the “Unlocking ML Performance on TPUs and GPUs” talk, which provides a walk through of how to use the aforementioned tools, a quick “tour” of XProf tools, and some real performance optimization scenarios using XProf on TPUs and GPUs.

Acknowledgements

This work constituted a massive effort across Google Cloud, CoreML as well as multiple teams within Google. Special thanks to Kan Cai, Vaibhav Tyagi, Victor Geislinger, Pavel Dournov, Navid Khajouei, Clive Verghese, Yin Zhang, Kelvin Le, Matt Hurd, Mudit Gokhale, Sai Ganesh, Vikas Aggarwal, Sannidhya Chauhan, Stephanie Morton, Subham Soni, Ani Udipi, Newfel Harrat, Aspi Siganporia, Bryan Massoth, Chetna Jain, David Duan, George Vanica, Jiten Thakkar, Lei Zhang, Jiya Zhang, Jonah Weaver, Aditya Sharma, Fenghui Zhang, Bill Jia, Alex Spiridonov, and Niranjan Hira for their immense contributions and support to these products.