Cloud

The AI state of the art is shifting rapidly from simple chat interfaces to autonomous agents capable of planning, executing, and refining complex workflows. In this new landscape, the ability to ground these intelligent agents in your enterprise data is key to unlocking true business value. Google Cloud is at the forefront of this shift, empowering you to build robust, data-driven applications quickly and accurately.

Last month, Google announced Antigravity, an AI-first integrated development environment (IDE). And now, you can now give the AI agents you build in Antigravity direct, secure access to the trusted data infrastructure that powers your organization, turning abstract reasoning into concrete, data-aware action. With Model Context Protocol (MCP) servers powered by MCP Toolbox for Databases now available within Antigravity, you can securely connect your AI agents to services like AlloyDB for PostgreSQL, BigQuery, Spanner, Cloud SQL, Looker and others within Google’s Data Cloud, all within your development workflow.

Why use MCP in Antigravity?

We designed Antigravity to keep you in the flow, but the power of an AI agent is limited by what it “knows.” To build truly useful applications, your agent needs to understand your data. MCP acts as the universal translator. You can think of it like a USB-C port for AI. It allows the LLMs in your IDE to plug into your data sources in a standardized way. By integrating pre-built MCP servers directly into Antigravity, you don’t need to perform any manual configuration. Your agents can now converse directly with your databases, helping you build and iterate faster without ever leaving the IDE.

Getting started with MCP servers

In Antigravity, connecting an agent to your data is a UI-driven experience, eliminating the challenges we’ve all faced when wrestling with complex configuration files just to get a database connection running. Here’s how to get up and running.

1. Discover and launch

You can find MCP servers for Google Cloud in the Antigravity MCP Store. Search for the service you need, such as “AlloyDB for PostgreSQL” or “BigQuery,” and click on Install to start the setup process.

2. Configure your connection

Antigravity presents a form where you can add your service details such as Project ID and region. You can also enter your password or have Antigravity use your Identity and Access Management (IAM) credentials for additional security. These are stored securely, so your agent can access the tools it needs without exposing raw secrets in your chat window.

See your agents in action

Once connected to Antigravity, your agent gains a suite of “tools” (executable functions) that it can use to assist you, and help transform your development and observability experience across different services. Let’s take a look at a couple of common scenarios. 

Streamlining database tasks with AlloyDB for PostgreSQL

When building against a relational database like PostgreSQL, you may spend time switching between your IDE and a SQL client to check schema names or test queries. With the AlloyDB MCP server, your agent handles that context and gains the ability to perform database administration and generate high-quality SQL code you can include in your apps — all within the Antigravity interface.

For example:

• Schema exploration: The agent can use list_tables and get_table_schema to read your database structure and explain relationships to you instantly.
• Query development: Ask the agent to “Write a query to find the top 10 users,” and it can use execute_sql to run it and verify the results immediately.
• Optimization: Before you commit code, use the agent to run get_query_plan to ensure your logic is performant.

Unlocking analytics with BigQuery

For data-heavy applications, your agent can act as a helpful data analyst. Leveraging the BigQuery MCP server, it can, for example:

• Forecast: Use forecast to predict future trends based on historical data.

• Search the catalog: Use search_catalog to discover and manage data assets.

• Augmented analytics: Use analyze_contribution to understand the impact of different factors on data metrics.

Building on truth with Looker

Looker acts as your single source of truth for business metrics. Looker’s MCP server allows your agent to bridge the gap between code and business logic, for example:

• Ensuring metric consistency: No more guessing whether a field is named total_revenue or revenue_total. Use get_explores and get_dimensions to ask your agent, “What is the correct measure for Net Retention?” and receive the precise field reference from the semantic model.

• Instantly validating logic: Don’t wait to deploy a dashboard to test a theory. Use run_query to execute ad-hoc tests against the Looker model directly in your IDE, so that your application logic matches the live data.

• Auditing reports: Use run_look to pull results from existing saved reports, allowing you to verify that your application’s output aligns with official business reporting.

Build with data in Antigravity

By integrating Google’s Data Cloud MCP servers into Antigravity, it’s easier than ever to use AI to discover insights and develop new applications. Now, with access to a wide variety of data sources that run your business, get ready to take the leap from simply talking to your code, to creating new experiences for your users.

To get started, check out the following resources:

• Documentation: Connecting to AlloyDB using MCP

• GitHub: MCP Toolbox for Databases

Building Generative AI applications has become accessible to everyone, but moving those applications from a prototype to a production-ready system requires one critical step: Evaluation.

How do you know if your LLM is safe? How do you ensure your RAG system isn’t hallucinating? How do you test an agent that generates SQL queries on the fly?

At its core, GenAI Evaluation is about using data and metrics to measure the quality, safety, and helpfulness of your system’s responses. It moves you away from “vibes-based” testing (just looking at the output) to a rigorous, metrics-driven approach using tools like Vertex AI Evaluation and the Agent Development Kit (ADK).

To guide you through this journey, we have released four hands-on labs that take you from the basics of prompt testing to complex, data-driven agent assessment.

Evaluating Single LLM Outputs

Before you build complex systems, you must understand how to evaluate a single prompt and its response. This lab introduces you to GenAI Evaluation, a service that helps you automate the evaluation of your model’s outputs.

You will learn how to define metrics, such as safety, groundedness, and instruction following. You will also learn how to run evaluation tasks against a dataset. This is the foundational step for any production-ready AI application.

Evaluate RAG Systems with Vertex AI

Retrieval Augmented Generation (RAG) is a powerful pattern, but it introduces new failure points: did the search fail to find the document, or did the LLM fail to summarize it?

This lab takes you deeper into the evaluation lifecycle. You will learn how to verify “Faithfulness” (did the answer come from the context?) and “Answer Relevance” (did it actually answer the user’s question?). You will pinpoint exactly where your RAG pipeline needs improvement.

Evaluating Agents with ADK

Agents are dynamic; they choose tools and plan steps differently based on the input. This makes them harder to test than standard prompts. You aren’t just grading the final answer; you are grading the trajectory, which is the path the agent took to get there.

This lab focuses on using the Agent Development Kit (ADK) to trace and evaluate agent decisions. You will learn how to define specific evaluation criteria for your agent’s reasoning process and how to visualize the results to ensure your agent is using its tools correctly.

Build and Evaluate BigQuery Agents

When an agent interacts with data, precision is paramount. A SQL-generating agent must write syntactically correct queries and retrieve accurate numbers. A hallucination here doesn’t just look bad, it might lead to bad business decisions.

In this advanced lab, you will build an agent capable of querying BigQuery and then use the GenAI Eval Service to verify the results. You will learn to measure Factual Accuracy and Completeness, ensuring your agent provides the exact data requested without omission.

Trust Your AI in Production

Ready to make your AI applications production-grade? Start evaluating your model’s outputs or the trajectory taken by your agents with these codelabs:

• Evaluating Single LLM Outputs With Vertex AI Evaluation

• Evaluate RAG Systems with Vertex AI

• Evaluating Agents with ADK

• Build and Evaluate BigQuery Agents using ADK and GenAI Eval Service

From Prototype to Production

These labs are part of the AI Evaluation module in our official Production-Ready AI with Google Cloud program. Explore the full curriculum for more content that will help you bridge the gap from a promising prototype to a production-grade AI application.

To build a production-ready agentic system, where intelligent agents can freely collaborate and act, we need standards and shared protocols for how agents talk to tools and how they talk to each other.

In the Agent Production Patterns module in the Production-Ready AI with Google Cloud Learning Path, we focus on interoperability, exploring the standard patterns for connecting agents to data, tools and each other. Here are three hands-on labs to help you build these skills.

The Foundations of ADK, MCP, A2A

This lab serves as your “Hello World” for the modern agent stack. You will build a simple, specialized agent that demonstrates how Agent Development Kit (ADK), Model Context Protocol (MCP), and Agent to Agent Protocol (A2A) work together. 

Connecting to Data with MCP

Once you understand the basics, the next step is giving your agent access to knowledge. Whether you are analyzing massive datasets or searching operational records, the MCP Toolbox provides a standard way to connect your agent to your databases.

Expose a BigQuery database to an MCP Client

This lab shows you how to expose BigQuery tables to an MCP client.

Expose a CloudSQL database to an MCP Client

If you need your agent to search for specific records—like flight schedules or hotel inventory—this lab demonstrates how to connect to a CloudSQL relational database.

From Prototype to Production

By moving away from custom integrations and adopting standards like MCP and A2A, you can build agents that are easier to maintain and scale. These labs provide the practical patterns you need to connect your agents to your data, your tools, and each other.

These labs are part of the Agent Production Patterns module in our official Production-Ready AI with Google Cloud Learning Path. Explore the full curriculum for more content that will help you bridge the gap from a promising prototype to a production-grade AI application.

Share your progress using the hashtag #ProductionReadyAI. Happy learning!

We can all agree that the quality of AI-driven answers relies on the consistency of the underlying data. But AI models, while powerful, lack business context out of the box. As more organizations ask questions of their data using natural language, it is increasingly important to unify business measures and dimensions, ensure consistency company-wide. If you want trustworthy AI, what you need is a semantic layer that acts as the single source of truth for business metrics.But how do you make that data accessible and actionable for your end users? Building off the recent introduction of Looker’s Model Context Protocol (MCP) server, in this blog we take you through the process of creating an Agent Development Kit (ADK) agent that is connected to Looker via the MCP Toolbox for Databases and exposing it within Gemini Enterprise. Let’s get started. Step 1 – Set up Looker Integration in MCP Toolbox

MCP Toolbox for Databases is a central open-source server that hosts and manages toolsets, enabling agentic applications to leverage Looker’s capabilities without working directly with the platform. Instead of managing tool logic and authentication themselves, agents act as MCP clients and request tools from the Toolbox. The MCP Toolbox handles all the underlying complexities, including secure connections to Looker, authentication and query execution.

The MCP Toolbox for Databases natively supports Looker’s pre-built toolset. To access these tools, follow the below steps:

Connect to Cloud Shell. Check that you’re already authenticated, and that the project is set to your project ID using the following command:

Run the following command in Cloud Shell to confirm that the gcloud command knows about your project:

Create a folder named mcp-toolbox:

Go to the mcp-toolbox folder via the command shown below:

Install the binary version of the MCP Toolbox for Databases via the script given below. This command is for Linux; if you run on Macintosh or Windows, ensure that you download the correct binary. Check out the releases page for your Operation System and Architecture and download the correct binary.

Deploy Toolbox to Cloud Run

Next, you’ll need to run MCP Toolbox. The simplest way to do that is on Cloud Run, Google Cloud’s fully managed container application platform. Here’s how:

Enable relevant APIs if needed:

Create a service account and assign necessary roles:

For information on getting your client id and secret, read the documentation.

Create a deploy.env file:

Export the toolbox config into an image variable:

Deploy MCP Toolbox to Cloud Run:

The Cloud Run will ask if you want Unauthenticated, select No.Allow Unauthenticated: N

Step 2: Deploy ADK Agent to Agent Engine

Next, you need to configure Agent Development Kit (ADK), a flexible and modular framework for developing and deploying AI agents. ADK was designed to make agent development feel more like software development, to make it easier for developers to create, deploy, and orchestrate agentic architectures that range from simple tasks to complex workflows. And while ADK is optimized for Gemini and the Google ecosystem, it’s also model-agnostic, deployment-agnostic, and is built for compatibility with other frameworks. 

Vertex AI Agent Engine, a part of the Vertex AI Platform, is a set of services that enables developers to deploy, manage, and scale AI agents in production. Agent Engine handles the infrastructure to scale agents in production so you can focus on creating applications.

Open a new terminal tab in Cloud Shell and create a folder named my-agents as follows. You also need to navigate to the my-agents folder.

Now, create a virtual Python environment using venv:

Activate the virtual environment:

Install the ADK and the MCP Toolbox for Databases packages along with langchain dependency:

Creating your first Agent Application

Now you’re ready to use adk to create a scaffolding, including folders, environment and basic files, for our Looker Agent Application via the adk create command with an app name looker_app:

Follow the steps and select the following:

• Gemini model for choosing a model for the root agent

• Vertex AI for the backend

• Your default Google Project Id and region

Observe the folder in which a default template and required files for the Agent have been created.

First up is the .env file:

The values indicate that you will be using Gemini via Vertex AI along with the respective values for the Google Cloud Project Id and location.

Then you have the __init__.py file that marks the folder as a module and has a single statement that imports the agent from the agent.py file:

Finally, take a look at the agent.py file. The contents can be edited to similar to the example below:

Insert the Cloud Run URL highlighted here (. not the one with the project number in the url).

Create a Cloud Storage bucket if necessary:

Ensure you are in the my-agents directory. Update the BUCKET_NAME with the Cloud Storage bucket you want to use:

NOTE: Ensure you grant the Cloud Run Invoker role to the default Agent Engine Service Account (i.e., service-PROJECT_NUMBER@gcp-sa-aiplatform-re.iam.gserviceaccount.com)

Step 3: Connect to Gemini Enterprise

Now it’s time to create a Gemini Enterprise app (instructions here).

Run the below command with the GCP Project Number, Reasoning Engine resource name output from the ‘deploy agent_engine’ command above, and your Gemini Enterprise Agent ID from the Gemini Enterprise Apps interface:

Your Looker data will now be available within your Gemini Enterprise app.If you don’t have access to this feature, contact your Google Cloud account team.

Querying business data made easier

Connecting Looker’s semantic layer to Vertex AI Agent services by way of the ADK and MCP Toolbox is a big win for data accessibility. By exposing your trusted Looker models and Explores in Gemini Enterprise, you empower end-users to query complex business data using natural language. This integration closes the gap between data insights and immediate action, ensuring that your organization’s semantic layer is not just a source of passive reports, but an active, conversational, and decision-driving asset.

To get started, connect to the MCP Toolbox for Databases on GitHub and set up Looker integration.

Today, many organizations operate with data that’s trapped in silos, in disconnected legacy systems and is days or hours old. However, the rise of AI presents the need and opportunity to unify these environments,  tap into unstructured data from audio, video, and text files, which together, makes up more than 80% of enterprise data and enable business decisions informed by real-time data. Data teams navigating AI also face a new set of challenges such as automating complex workflows and apps, grounding them in enterprise data, activating real-time insights on multimodal data, and building a foundation that inspires trust in AI. 

Google’s Data Cloud is an AI-native platform designed to unify an organization’s entire data foundation and enable intelligent applications and agentic experiences. Data Cloud integrates Google infrastructure, intelligence, and data platform with pioneering AI advancements, including Gemini for working with data, automation of metadata management and governance, and flexible workflows for developers, allowing customers to focus on innovation and business outcomes rather than integration challenges. 

Recently, we were honored to be recognized as a Leader in the 2025 Gartner® Magic Quadrant™ for Data Integration Tools. In our opinion this demonstrates Data Cloud’s tight integration with data integration tools and vision for AI, including customer use cases for multimodal data processing, and scalable, efficient vectorization. In addition, we were recognized as a leader in the Forrester Wave™ :Streaming Data Platforms, Q4 2025. In this blog post, we take a look at recent updates and innovations that we believe made recognition from these two leading analyst firms possible.   

Boost productivity with Gemini-powered intelligence

Data agents are revolutionizing the way different data roles operate by bringing automation, intelligence, and natural language capabilities into their daily workflows. Whether you’re a data analyst querying and visualizing data more efficiently, a developer building smarter applications, or a data scientist accelerating model development, agents can help streamline repetitive tasks and boost your productivity. Data engineers benefit from automated data preparation and pipeline management, while ML engineers can deploy and monitor models more effectively. Even business users, who traditionally rely on technical teams for insights, can now interact with data directly using natural language. 

Recent innovations to Gemini with BigQuery for data engineering provide automation to build data pipelines to ingest, transform, and validate data. This includes data transformations like data cleaning, deduplication, formatting, standardizing, joins, and aggregations as well as data quality to enforce rules and standards. Building on these capabilities, the Data Engineering Agent further accelerates productivity by intelligently automating these standard integration patterns and proactively monitoring pipeline health.

Speed efficiency with multimodal automation and governance

We are removing the friction to build AI applications using autonomous vector embedding for multimodal data. Building on our BigQuery Vector Search capabilities, data teams can build, manage, and maintain complex data pipelines without needing to update vector embeddings. BigQuery now takes care of this automatically with added capabilities for agents to connect user intent to enterprise data. This is powering customer systems like the in-store product finder at Morrisons, which handles 50,000 customer searches on a busy day.

We are also helping organizations ensure their data platform acts as a real-time brain for AI, including orchestration and AI-infused services. Governance is foundational to data and AI success. In today’s world of distributed data spanning lakes, warehouses, and operational systems, intelligence is impossible without unified governance. 

New automated cataloging with Dataplex Universal Catalog allows data teams to discover, ingest, and index metadata from a wide range of sources, minimizing the effort involved in cataloging data, and providing a near-real-time view of your data and AI landscape. Dataplex provides context to your data teams and your agents beyond the normal scope of a universal catalog.  It leverages Gemini to continuously derive relationships and auto-generate business semantics, providing AI agents with trusted, real-time context. 

Ericsson uses Dataplex to deliver a unified business vocabulary to users, including data classification, ownership, retention policies, and sensitivity labels. This allows different data personas to instantly understand a data origin, increasing trust and reducing investigation time. 

Optimize workloads for broad usability 

Managing data across cloud and hybrid environments can be piecemeal, leading to costly inefficiencies, redundant storage, and complex data movement.

To help, visual pipelines provide a code-free user experience for designing, deploying, managing and monitoring pipelines, with a metadata-driven approach to improving developer productivity. And enhancements to data preparation in BigQuery provide a single platform to clean, structure, enrich and build data pipelines. 

For ML transformations supporting retrieval augmented generation (RAG) use cases, recent innovations enhance model inference to ML models in real-time or batch. And support for libraries and frameworks for multimodal data allows data teams to leverage multiple models in a single pipeline, improving accuracy and recall.

Integrating real-time data and context for AI

Agents need context in order to be effective and are significantly limited when they rely on static or outdated information. To make accurate decisions that genuinely help users and the business, they need real-time access to the current state of your systems and users. We launched Managed Service for Apache Kafka last year to help you integrate your operational and transactional data into your AI and data platform that in turn can then power your AI agents. This year, we added critical enterprise capabilities such as Apache Kafka Connect, VPC Service Controls, mutual TLS authentication, and Kafka access control which have helped customers like MadHive deploy to production in a matter of months. To enable new streaming architectures, we added User-Defined Functions support (UDFs) in Pub/Sub for transforming messages (like JSON) before they go to destinations like BigQuery, allowing custom logic, validation, and enrichment on the streaming data and making Pub/Sub pipelines more powerful and flexible. We also enhanced Dataflow, the advanced unified streaming and batch processing engine with critical capabilities such as parallel updates, Managed I/O, Google Cloud TPU support, speculative execution and more to bring the power of AI enabled data processing to advanced stream processing use cases such as continuous ML feature extraction and real time fraud detection.

Data integration and streaming momentum

It was a busy year for the Google Data Cloud team, and we are honored to be recognized in these recent Gartner and Forrester reports. We look forward to continuing to innovate and partner with you on your data transformation journey.

_{Forrester does not endorse any company, product, brand, or service included in its research publications and does not advise any person to select the products or services of any company or brand based on the ratings included in such publications. Information is based on the best available resources. Opinions reflect judgment at the time and are subject to change. For more information, read about Forrester’s objectivity here .}

_{Gartner, Magic Quadrant for Data Integration Tools, Michele Launi, Nina Showell, Robert Thanaraj, Sharat Menon, 8 December 2025}

_{Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.}

_{GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, and MAGIC QUADRANT is a registered trademark of Gartner, Inc. and/or its affiliates and are used herein with permission. All rights reserved.}

Today, we are excited to announce that Gemini Live API, powered by the latest Gemini 2.5 Flash Native Audio model, is generally available on Vertex AI.

Pioneering organizations have been using Gemini Live API to build the next generation of multimodal conversational AI that blends voice, vision, and text, to deliver fluid, human-like, and highly contextual interactions. For Google Cloud customers, this means you can deploy low-latency voice and video agents with the stability and performance required for your most demanding workflows.

A new standard with real-time multimodal AI agents

Gemini Live API represents a new standard for bringing AI to life. Imagine an agent that doesn’t just listen, but instantly understands the user’s intent, the context of their screen, captures the emotion in their voice, and responds with a human-like voice — all in real time.

The power behind this dynamic capability is the Gemini 2.5 Flash Native Audio model. Our approach is based on a simple commitment: to bring the same high-quality conversational intelligence found in advanced experiences across Google directly to your enterprise applications.

In a real-time interaction, precision and speed are non-negotiable. Gemini Live API is natively multimodal and is designed to handle the instantaneous complexity of human dialogue:

• It can process interruptions mid-sentence without missing a beat, ensuring natural turn-taking.

• It understands acoustic cues like pitch and pace, deciphering intent and tone.

• It can see and discuss complex visual data (charts, live video, diagrams) shared by a user, providing immediate, contextual assistance.

The confidence to deploy on Vertex AI

Gemini Live API is engineered for enterprise success. Vertex AI provides the security and stability your mission-critical agents need for production.

The Gemini 2.5 Flash Native Audio model is optimized to process a high volume of concurrent interactions with consistent, low-latency performance. Deploying on Vertex AI allows you to leverage our expanding global infrastructure across multiple regions, delivering reliability for your users. Additionally, enterprise-grade data residency features that allow you to manage where your data is processed, helping you meet critical regulatory and compliance standards. 

Building real-world impact with Gemini Live API

The true power of Gemini Live API is demonstrated by the companies who are using it today to redefine their customer experiences.

Shopify, the leading global commerce platform, developed Sidekick, a multimodal AI assistant powered by Gemini Live API on Vertex AI. It provides personalized, robust support away from a desk, enabling real-time problem solving that eliminates traditional ticketing workflows.

“Users often forget they’re talking to AI within a minute of using Sidekick, and in some cases have thanked the bot after a long chat. This is an exciting time to be an entrepreneur. New AI capabilities offered through Gemini empower our merchants to win.” – David Wurtz, VP of Product, Shopify

United Wholesale Mortgage (UWM) transformed its business process by using their AI Loan Officer Assistant, Mia, to dramatically increase business efficiency for their broker partners.

“By integrating the Gemini 2.5 Flash Native Audio model and harnessing the Gemini Live API capabilities on the Vertex AI platform, we’ve significantly enhanced Mia’s capabilities since launching in May 2025. This powerful combination has enabled us to generate over 14,000 loans for our broker partners, proving that AI is much more than just a buzzword at UWM.” – Jason Bressler, Chief Technology Officer, UWM

SightCall provides remote video support and AI-driven visual assistance, helping customer service and field teams solve problems faster. 

“What makes this partnership so exciting is that the Gemini 2.5 Flash Native Audio model isn’t just fast — it’s seamlessly human. When combined with SightCall Xpert Knowledge™, it becomes a real-time expert that knows what your best technicians know… This is the future of visual support.” – Thomas Cottereau, CEO, SightCall

Napster uses the Gemini Live API’s vision and audio capabilities so their users can co-create and receive live guidance from specialized AI companions.

“By utilizing the Gemini 2.5 Flash Native Audio model on Vertex AI, we’ve built something we couldn’t before: AI Companions that see you, see your screen, and respond like real experts in real-time conversation. This combination of vision and audio enables genuine collaboration — no prompting, no engineering — just natural dialogue where AI understands your full context and unlocks creativity and expertise for everyone.” – Edo Segal, CTO, Napster

Lumeris is deploying their health AI assistant, Tom, in high-stakes environments where nuance and emotional sensitivity are non-negotiable.

“The transition to the Gemini Live API on Vertex AI is a strategic investment in more intuitive and efficient patient conversations. The result is a more responsive and personalized voice experience. For Lumeris, our goal is elevating the quality of every interaction between patients and Tom, our agentic primary care team member. This helps us set a new standard for patient care.” – Jean-Claude Saghbini, President and Chief Technology Officer, Lumeris

Newo deploys versatile AI Receptionists that achieve a conversational quality that is truly lifelike and emotionally intuitive, handling tasks from general inquiries to sales.

“Working with the Gemini 2.5 Flash Native Audio model through Vertex AI allows Newo.ai AI Receptionists to achieve unmatched conversational intelligence — combining ultra-low latency with advanced reasoning. They can identify the main speaker even in noisy settings, switch languages mid-conversation, and sound remarkably natural and emotionally expressive. Our Gemini Live API-powered outbound AI Sales Agents can laugh, joke, and truly connect — making every call feel human.” – David Yang, co-founder, Newo.ai

11Sight is redefining customer interactions with AI-powered conversational agents that book appointments and close sales.

“The Gemini 2.5 Flash Native Audio model on Vertex AI gave us the enterprise-grade platform required to rapidly develop our voice AI agents with very low latency. Integrating this solution with our Sentinel AI Agents pushed our call resolution rates from 40% in February to 60% in November.” – Dr. Farokh Eskafi, CTO, 11Sight

Start building your next-generation agent today

Here is how you can start building with Gemini Live API on Vertex AI today: 

• Try Gemini Live API now in Vertex AI Studio.

• Read the developer blog to dive deeper into creative use cases, code snippets, and a step-by-step guide for implementation.

• Explore our Gemini Live API documentation for API specifics, reference architectures, and even more demos.

Give your AI apps and agents a natural, almost human-like interface, all through a single WebSocket connection. 

Today, we announced the general availability of Gemini Live API on Vertex AI, which is powered by the latest Gemini 2.5 Flash Native Audio model. This is more than just a model upgrade; it represents a fundamental move away from rigid, multi-stage voice systems towards a single, real-time, emotionally aware, and multimodal conversational architecture.

We’re thrilled to give developers a deep dive into what this means for building the next generation of multimodal AI applications. In this post we’ll look at two templates and three reference demos that help you understand how to best use Gemini Live API.

Gemini Live API as your new voice foundation

For years, building conversational AI involved stitching together a high-latency pipeline of Speech-to-Text (STT), a Large Language Model (LLM), and Text-to-Speech (TTS). This sequential process created the awkward, turn-taking delays that prevented conversations from ever feeling natural.

Gemini  Live API fundamentally changes the engineering approach with a unified, low-latency, native audio architecture.

• Native audio processing: Gemini 2.5 Flash Native Audio model processes raw audio natively through a single, low-latency model. This unification is the core technical innovation that dramatically reduces latency.
• Real-time multimodality: The API is designed for unified processing across audio, text, and visual modalities. Your agent can converse about topics informed by live streams of visual data (like charts or live video feeds shared by a user) simultaneously with spoken input.

Next-generation conversation features

Gemini Live API  gives you a suite of production-ready features that define a new standard for AI agents:

• Affective dialogue (emotional intelligence): By natively processing raw audio, the model can interpret subtle acoustic nuances like tone, emotion, and pace. This allows the agent to automatically de-escalate stressful support calls or adopt an appropriately empathetic tone.
• Proactive audio (smarter barge-in): This feature moves beyond simple Voice Activity Detection (VAD). As demonstrated in our live demo, you can configure the agent to intelligently decide when to respond and when to remain a silent co-listener. This prevents unnecessary interruptions when passive listening is required, making the interaction feel truly natural.
• Tool use: Developers can seamlessly integrate tools like Function Calling and Grounding with Google Search into these real-time conversations, allowing agents to pull real-time world knowledge and execute complex actions immediately based on spoken and visual input.
• Continuous memory: Agents maintain long, continuous context across all modalities.
• Enterprise-grade stability: With GA release, you get the high availability required for production workloads, including multi-region support to ensure your agents remain responsive and reliable for users globally.

Developer quickstart: Getting started

For developers, the quickest way to experience the power of low-latency, real-time audio is to understand the flow of data. Unlike REST APIs where you make a request and wait, Gemini Live API requires managing a bi-directional stream.

Gemini Live API flow

Before diving into code, it is critical to visualize the production architecture. While a direct connection is possible for prototyping, most enterprise applications require a secure, proxied flow: User-facing App -> Your Backend Server -> Gemini Live API (Google Backend).

In this architecture, your frontend captures media (microphone/camera) and streams it to your secure backend, which then manages the persistent WebSocket connection to Gemini Live API in Vertex AI. This ensures sensitive credentials never leave your server and allows you to inject business logic, persist conversation state, or manage access control before data flows to Google.

​​To help you get started, we have released two distinct Quickstart templates – one for understanding the raw protocol, and one for modern component-based development.

Option A: Vanilla JS Template (zero dependency)

Best for: Understanding the raw WebSocket implementation and media handling without framework overhead.

This template handles the WebSocket handshakes and media streaming, giving you a clean slate to build your logic.

Project Structure:

Core implementation: You interact with the gemini-live-2.5-flash-native-audio model via a stateful WebSocket connection.

Running the Vanilla JS Demo:

Follow along the step-by-step video walkthrough.

Pro-tip: Debugging raw audio Working with raw PCM audio streams can be tricky. If you need to verify your audio chunks or test Base64 strings, we’ve included a PCM Audio Debugger in the repository.

Option B: React demo (modular & modern)

Best for: Building scalable, production-ready applications with complex UIs.

If you are building a robust enterprise application, our React starter provides a modular architecture using AudioWorklets for high-performance, low-latency audio processing.

Features:

• Real-time streaming: Audio and video streaming to Gemini with React state management.
• AudioWorklets: Uses capture.worklet.js and playback.worklet.js for dedicated audio processing threads.
• Secure proxy: Python backend handles Google Cloud authentication.

Project structure:

Running the react demo:

Follow along the step-by-step video walkthrough.

Partner Integrations

If you prefer a simpler development process for specific telephony or WebRTC environments, we have third-party partner integrations with Daily, Twilio, LiveKit, and Voximplant. These platforms have integrated the Gemini Live API over the WebRTC protocol, allowing you to drop these capabilities directly into your existing voice and video workflows without managing the networking stack yourself .

Gemini Live API: Three production-ready demos

Once you have your foundation set with either template, how do you scale this into a product? We’ve built three demos showcasing the distinct “superpowers” of Gemini Live API.

1. Real-time proactive advisor agent

The core of building truly natural conversational AI lies in creating a partner, not just a chatbot. This specialized application demonstrates how to build a business advisor that listens to a conversation and provides relevant insights based on a provided knowledge base.

It showcases two critical capabilities for professional agents: Dynamic Knowledge Injection and Dual Interaction Modes.

• The Scenario: An advisor sits in on a business meeting. It has access to specific injected data (revenue stats, employee counts) that the user defines in the UI.

• Dual modes:

• Silent mode: The advisor listens and “pushes” visual information via a show_modal tool without speaking. This is perfect for unobtrusive assistance where you want data, not interruption.

• Outspoken mode: The advisor politely interjects verbally to offer advice, combining audio response with visual data.

Check out the full source code for the real-time advisor agent implementation in our GitHub repository.

2. Multimodal customer support agent

Customer support agents must be able to act on what they “see” and “hear.” This demo layers Contextual Action and Affective Dialogue onto the voice stream, creating a support agent that can resolve issues instantly.

This application simulates a futuristic customer support interaction where the agent can see what you see, understand your tone, and take real actions to resolve your issues instantly. Instead of describing an item for a return, the user simply shows it to the camera. The agent combines this visual input with emotional understanding to drive real actions:

• Multimodal Understanding: The agent visually inspects items shown by the customer (e.g., verifying a product for return) while listening to their request.
• Empathetic Response: Using affective dialogue, the agent detects the user’s emotional state (frustration, confusion) and adjusts its tone to respond with appropriate empathy.
• Action Taking and Tool Use: It doesn’t just chat; it uses custom tools like process_refund (handling transaction IDs) or connect_to_human (transferring complex issues) to actually solve the problem.
• Real-time Interaction: Low-latency voice interaction using Gemini Live API over WebSockets.

Check out the full source code for the multi-modal customer support agent implementation in our GitHub repository.

3. Real-time video game assistant

Gaming is better with a co-pilot. In this demo, we build a Real-Time Gaming Guide that moves beyond simple chat to become a true companion that watches your gameplay and adapts to your style.

This React application streams both your screen capture and microphone audio to the model simultaneously, allowing the agent to understand the game state instantly. It showcases three advanced capabilities:

• Multimodal awareness: The agent acts as a second pair of eyes, analyzing your screen to spot enemies, loot, or puzzle clues that you might miss.
• Persona switching: You can dynamically toggle the agent’s personality – from a “Wise Wizard” offering cryptic hints to a “SciFi Robot” or “Commander” giving tactical orders. This demonstrates how system instructions can instantly change the voice and style of assistance.
• Google Search Grounding: The agent pulls real-time information to provide up-to-date walkthroughs and tips, ensuring you never get stuck on a new level.

Check out the full source code for the real-time video game assistant implementation in our GitHub repository.

Get started today 

• Try it out today: Experiment with Gemini Live API in Vertex AI Studio
• Start building: Access Gemini Live API on Vertex AI today and move beyond chatbots to create truly intelligent, responsive, and empathetic user experiences.
• Get the code: All demos and quickstarts are available in our official GitHub repository.