Cloud

At Definity, a leading Canadian P&C insurer with a history spanning over 150 years, we have a long tradition of innovating to help our customers and communities adapt and thrive. To stay ahead in our rapidly evolving industry, we knew a unified data foundation was key to realizing the business and customer experience opportunities offered by modern analytics and AI. 

While our legacy on-premises Cloudera platform had served us well, it could no longer support our growing needs for scale, innovation, and harnessing the power of data and AI. So, we embarked on a critical mission: modernizing our data infrastructure.

Legacy limitations stifling innovation

We faced a combination of interconnected challenges, which impact many organizations today:

• Limited scalability and AI/ML workload support: Our existing infrastructure, constantly running at 80% utilization, was stretched thin. Processing billions of daily events for real-time analytics and scaling AI and ML workflows was a constant battle, limiting our ability to gain timely insights and develop innovative, data-driven products and experiences.

• Data silos, fragmented insights: Our data resided in various systems, creating a fragmented view of our business. This made it difficult to get a holistic understanding of our customers and hindered initiatives like building a comprehensive customer 360º view and delivering personalized recommendations at a moment of relevance.

• Escalating costs: Maintaining and scaling our Cloudera platform, which hosted massive data volumes (200TB compressed, 1PB uncompressed), was increasingly expensive and diverting valuable fiscal and people resources away from strategic priorities.

Faced with these pressing issues, the timing of our next renewal presented a strategic window of opportunity. We had a critical decision to make — migrate both technology and business platforms within 10 months or invest in upgrading our legacy Cloudera environment.

Building a unified data and AI platform with Google Cloud

We chose Google Cloud and its powerful duo, BigQuery and Vertex AI, to build the Strategic Data Platform (SDP) — our new modern data analytics platform. BigQuery’s serverless architecture, unmatched scalability, built-in ML capabilities, and seamless integration with Vertex AI made it the ideal solution to power our data-driven transformation.Our migration was a remarkably fast-paced effort, carried out in close collaboration with with Google Cloud and Quantiphi, a Google Cloud partner. 

Like many enterprises, we adopted a hybrid approach. We retained Databricks on Google Cloud for specific ETL workloads, utilizing Quantiphi’s expertise in converting legacy systems. At the same time, we migrated the bulk of our data processing to BigQuery for optimal cost-efficiency and performance. We also used Cloud Composer to orchestrate our complex data pipelines and ensure secure, private connectivity within our Google Cloud environment, a crucial requirement for handling sensitive customer data.

As a result, our dedicated team of over 100 Definity employees completed the migration in just ten months — 50% faster than the industry average. This rapid transition was aided by innovative tools, such as the “nifi-migration” solution built by Google Cloud Consulting. This open-source tool provided a visual and highly configurable way to automate real-time data flow between different systems, minimizing disruption and helping us surpass our initial migration timeline expectations.

Our CTO, Tatjana Lalkovic, who championed this effort to consolidate our structured and multimodal data to accelerate our AI/ML use cases, shared her perspective on the impact of our decision, saying:

“As we reimagined where data and AI could take our business, industry, and customer experience, Google Cloud BigQuery and Vertex AI stood out as modern, enterprise-ready serverless solutions prepared to meet the AI moment — not just today but for the foreseen future. The speed and success of this migration has created a lot of trust in our partnership and has been a significant boost to our digital transformation to streamline operations, improve products, and better serve customers.”

Strategic Data Platform – High level design on Google Cloud

Transforming insurance with data and AI

The results of our migration to BigQuery and Vertex AI has been transformative for Definity. We’ve seen exceptional user satisfaction, with the SDP achieving a remarkable Net Promoter Score (NPS) of 9.9 out of 10. The move has also saved us millions on our annual spend on non-strategic technologies and delivered a roughly 75% reduction in planned downtime for our digital platforms. 

Performance has also dramatically improved, with processes to gain insights that once took days now completing in an average of 4.5 hours. Moreover, migrating to Google Cloud has helped us increase agility and innovation. We’re now able to double our business releases per year — achieving a 30% increase in testing automation, a 63% improvement in deployment time, and a 10x faster infrastructure setup. 

By combining structured and unstructured data in BigQuery, we’ve unlocked new analytical possibilities and improved price-performance. This unified data foundation has empowered our business intelligence tools with richer, more comprehensive data, leading to more informed business decisions. The seamless integration with Vertex AI has enabled us to develop, deploy, and scale AI models, driving innovation in areas like fraud detection, automated intake, and personalized call center experiences. At the same time, we benefit from Google Cloud’s strong commitment to data security and privacy, helping us to strengthen our security posture and keep our customers safe.

As our VP of Data, Ron Mills, said:

“BigQuery’s serverless architecture has been a game-changer. The ‘nothing to manage’ approach is a huge differentiator. For enterprises like us that are migrating from on-prem clusters constantly running at 80% capacity, it’s like night and day.”

Lessons learned from our migration journey

Migrating a core data platform is a significant undertaking, and we’ve learned a lot along the way. For other organizations considering the same journey, here are some key takeaways from our experience:

• One team, one goal: Foster a collaborative environment where technology and business teams, vendors, contractors, and consultants work together seamlessly towards a shared objective.

• Leadership trust and commitment: Executive leadership trust in the delivery team’s decision-making is crucial for maintaining momentum and navigating challenges.

• Be bold: Don’t be afraid to think outside the box, make timely decisions, and be prepared to adapt quickly to unforeseen setbacks.

• Plan for the unknown: Anticipate potential roadblocks and have a core team dedicated to developing alternative solutions and addressing unforeseen issues.

• Strong business partnership: A trusted relationship with business teams is essential for smooth user acceptance testing, change management, and avoiding unnecessary disruptions during the migration.

• Balanced governance: Independent governance should provide guidance and support calculated risk-taking, acting as a partner in problem-solving rather than a blocker.

• Motivated team: Cultivate a team-oriented environment where ownership of the project extends beyond leadership to every team member.

• Transparent communication: Maintain open and consistent communication among all stakeholders (in our case, over 250 people) to ensure everyone is aligned and informed.

• Fast fail and incremental delivery: Avoid a “big bang” approach. Embrace incremental releases (we aimed for 2-5 daily releases) to learn quickly, adapt, and iterate.

• Parallel run: Plan for a parallel run of your systems on both the legacy and target cloud platforms to ensure a smooth transition and validate the new environment.

A data-driven future with limitless potential

Our migration to BigQuery and Vertex AI is just the first step in Definity’s data transformation journey. With a modern, scalable, and AI-ready data foundation now in place, we are empowered to unlock even greater value from our data and continue to lead innovation in the insurance industry. We are excited about the possibilities that lie ahead and are already actively developing our next AI use cases, including several focused on legal summarization and IT functions. We are confident that our partnership with Google Cloud will be instrumental in helping us achieve our goals.

Learn how to start your own migration journey to BigQuery today.

Get ready to dive deep and level up your cloud skills at Google Cloud Next ’25.  Whether you’re a seasoned pro or just starting your cloud journey, you’ll have more learning opportunities at Next than ever before. From hands-on challenges to expert-led workshops on AI and ML, Next ’25 (April 9-11, 2025) is your chance to transform your knowledge into real-world expertise. 

Secure your ticket and register today.

The first-ever Skills Challenge: your chance to win big

This year, we’re launching a new, on-the-ground game: The Skills Challenge. Think of it as your personal learning adventure at Google Cloud Next, complete with:

• Hands-on labs: Master practical skills. 

• Certification kickstarters: Pave your way to certification.

• AI Agent Builder Bar: Experience AI agent development.

• Quizzathon at Makerspace: Test your Google Cloud knowledge and win.

• Leaderboard competition: See how you stack up against your peers and compete for grand prizes.

• Early access: Be the first to try new gamification features on Google Cloud Skills Boost.

Don’t miss out on limited-edition swag and bragging rights on the leaderboard displayed at the Learning and Certification booth. Top the charts by the third day of Next for a chance to win a grand prize.

Dive into expert-led workshops

Developers, these lab-style sessions are for you. They’re led by knowledgeable experts who can help you build better, faster, and smarter applications. Secure your spot today by adding training sessions to your agenda, such as:

• Get started with Vertex AI Studio

• Explore data with Gemini in BigQuery

• Build a chat application using the Gemini API on Cloud Run

Register today, as seats are going fast.

Gain insights from breakout sessions and lightning talks

From in-depth technical learning to strategic insights for leaders, explore topics like credentialing a modern workforce, and building a culture of continuous learning with expert-led presentations and panel discussions. Hear from customers about their technical use cases and gain valuable takeaways about their specific approaches and solutions.

• Session Day 1: Future-proof your AI skills: Credentialing the modern workforce
• Session Day 2: The ROI of continuous learning: Build and validate skills to close the AI skills gap
• Lightning talk Day 1: Google Launchpad: Building a Global Talent Pipeline for Cloud and AI 
• Lightning talk Day1: Riding the AI Wave Securely: Uber’s Training Program for Building Trust 

Recharge at the Google Developer Experts and Certified Lounge

Connect with your peers, recharge, and refuel at an exclusive lounge reserved for Google Developer Experts. Enjoy light refreshments, meetups, and a photowall to capture your Next ’25 experience. When you register for Next, remember to identify yourself as Google Cloud Certified on your Profile page for easy lounge entry!

Can’t wait? Start learning today with Google Cloud Skills Boost

Don’t wait until Next to start your learning journey. Jump-start your skills with courses, labs, learning paths and more on Google Cloud Skills Boost. Join the Innovators program to get 35 credits at no cost and use them to keep learning on Skills Boost. 

We’ll see you April 9-11 at Google Cloud Next ‘25! Register today.

Today’s insurance customers expect more: simple digital services, instant access to service representatives when they want to discuss personal matters, and quick feedback on submitted invoices. Meeting these demands has become increasingly difficult for insurers due to rising inquiry volumes, a shortage of skilled workers, and the loss of expertise as employees retire. 

Recognizing the growing need for immediate and accurate responses, SIGNAL IDUNA, a leading German full-service insurer, particularly prominent in health insurance, introduced a cutting-edge AI knowledge assistant, powered by Google Cloud generative AI.

“We’ve pioneered to unlock the power of human-AI collaboration: To redefine process efficiency by bringing together technology and subject matter experts to deliver exceptional customer experiences,” said Johannes Rath, board member for Customer, Service, and Transformation at SIGNAL IDUNA.

SIGNAL IDUNA, in collaboration with Google Cloud, BCG and Deloitte, has developed an AI knowledge assistant that empowers service agents to quickly and accurately resolve complex customer inquiries. This innovative solution uses Google Cloud AI, including Google’s multimodal Gemini models, to help agents find relevant documents and provide comprehensive answers 30% faster — ultimately, enhancing customer satisfaction.

The Challenge: Meeting modern expectations

Like many organizations in the insurance sector, SIGNAL IDUNA faced significant operational burdens. The complexity of insurance products, along with the growing demand for immediate and accurate responses, often leads to bottlenecks that can impact service experiences.

For example, prior to introducing its AI knowledge assistant, service agents had to manually search thousands of internal documents for hundreds of different tariffs to find the information needed to answer questions or resolve customer issues — including, insurance conditions, tariff information, guidelines, and standard operating procedures. As a result, 27% of inquiries required further escalation to other departments or specialists, resulting in delayed resolutions, increased costs, and potential damage to reputation.

Though complex, SIGNAL IDUNA prioritized this process as one of its top gen AI use cases, developing an AI assistant to help agents provide quick and accurate answers to customer inquiries, particularly those about health insurance. The AI knowledge assistant is grounded in more than 2,000 internal documents for more than 600 different tariffs, allowing agents to ask questions in natural language and receive accurate answers, significantly reducing the time spent searching for relevant information.

A deep dive into SIGNAL IDUNA’s gen AI system

Working with Google Cloud, BCG, and Deloitte, SIGNAL IDUNA built a sophisticated generative AI architecture using Google Cloud’s AI platform, Vertex AI, and utilized Gemini 1.5 Pro’s long-context capabilities to develop an AI knowledge assistant that can provide quick and accurate access to the right information within a vast collection of documents. The system employs multiple steps to aggregate and process extensive information from diverse sources, ensuring agents can access the complete context necessary to effectively address customer inquiries.

Here’s a breakdown of the key steps:

1. Data pre-processing and extraction
The knowledge base is built from various document types, which are typically in PDF format, including policy documents, operating procedures, and general terms and conditions.

SIGNAL IDUNA utilizes a hybrid approach that combines Layout Parser in Google Cloud Document AI and PDFPlumber to parse these PDFs and extract the text content. While the Layout Parser is responsible for extracting the text segments, SIGNAL IDUNA enhances the extraction of tables with PDFPlumber if the quality of the PDFs allows. The extracted texts are then cleaned, chunked by Google’s Gecko multilingual embedding model, and enhanced with additional metadata, enabling the ability to process and analyze the information later effectively.

For storing the vectorized texts, Google Cloud SQL for PostgreSQL is used with the pgvector PostgreSQL extension, which provides a highly effective vector database solution for our needs. By storing vectorized text chunks in Cloud SQL, SIGNAL IDUNA benefits from its scalability, reliability, and seamless integration with other Google Cloud services, while pgvector empowers efficient similarity search functionality.

2. Query augmentation
Query augmentation generates multiple queries to improve the formulation of user questions for both document retrieval from the vector store and answer generation. The original question is reformulated into several variants, creating three versions in total: the original query, a rewritten query, and an imitation query. These are used then to retrieve relevant documents and generate the final answer.

For the rewritten query, the system uses Gemini Pro 1.5 to correct spelling errors in the original question. Additionally, the query is expanded by adding synonyms for predefined terms and tagging specific terms (e.g., “remedies,” “assistive devices,” “wahlleistung/selective benefits”) with categories. The system also uses information about selected tariffs to enrich the query. For example, tariff attributes, such as brand or contract type, are extracted from a database and appended to the query in a structured format. These specific adjustments make it possible to handle special tariff codes and add further context based on tariff prefixes.

The imitation query uses Gemini Pro 1.5 to rephrase the question to mimic the language of technical insurance documents, improving the semantic similarity with the source material. It considers conversation history and handles age formatting.

3. Retrieval
First, the system checks the query cache, which stores previously answered questions and their corresponding correct answers. If the question, or one very similar to it, has already been successfully resolved, the cached answer is retrieved, helping to provide a rapid answer. This efficient approach ensures quick access to information and avoids redundant processing. 

The accuracy of the cache is maintained through a user feedback loop, which identifies correctly answered questions to be stored in the cache through upvotes. A downvote on a cached answer triggers an immediate cache invalidation, ensuring only relevant and helpful responses are served. This dynamic approach improves the efficiency and accuracy of the system over time. If no matching questions are found in the query cache, the retrieval process falls back on the vector store, ensuring that the system can answer novel questions.

After retrieving any relevant information chunks from the query cache or vector store, the system uses the Vertex AI ranking API to rerank them. This crucial process analyzes various signals to refine the results, prioritizing relevance and ensuring the most accurate and helpful information is presented.

Ensuring complete and accurate answers is paramount during retrieval, and SIGNAL IDUNA found that some queries required information beyond what was available in the source documents. To address this issue, the system uses keyword-based augmentations to supplement the final prompt, providing a more comprehensive context for generating responses.

4. Generation
The answer generation process involves three key components: the user’s question with multiple queries, retrieved chunks of relevant information, and augmentations that add further context. These elements are combined to create the final response using a complex prompt template.

Delivering a near real-time experience is crucial for service agents, so SIGNAL IDUNA also streams the generated response. During development, minimizing latency based on the input posed a significant technical hurdle. To address this issue, SIGNAL IDUNA reduced processing times using asynchronous APIs to help stream data and handle multiple requests. Currently, the system has achieved an average response time of approximately 6 seconds, and SIGNAL IDUNA is experimenting with newer faster models to reduce this time even further.

5. Evaluation
Rigorous evaluation is essential for optimizing Retrieval Augmented Generation (RAG) systems. SIGNAL IDUNA uses the Gen AI evaluation service in Vertex AI to automate the assessment of both response quality and the performance of all process components, such as retrieval. A comprehensive question set, created with input from SIGNAL IDUNA’s service agents, forms the basis of these automated tests. 

The evaluation results flow seamlessly into Vertex AI Experiments and Google Cloud BigQuery. This enables SIGNAL IDUNA to visualize performance trends and gain actionable insights using dashboards with Looker on Google Cloud.

Here’s a closer look at how Looker helps evaluate the AI knowledge assistant:

• Chunk retrieval: First, SIGNAL IDUNA evaluates retrieval of relevant information chunks. Metrics at this stage help assess how effectively the model identifies and gathers the necessary information from the source data. This includes tracking gen AI metrics, such as recall, precision, and F1-score, to pinpoint areas for improvement in the retrieval process. This is crucial as retrieving the correct information is the foundation of a good generated response.

• Document reranking: Once the relevant chunks are retrieved, they’re reranked to prioritize the most pertinent information. The Looker dashboard allows monitoring the effectiveness of this reranking process.

• Generated vs. expected response comparison: The final stage involves comparing the generated response with the expected response. SIGNAL IDUNA evaluates the quality, accuracy, and completeness of the generated output, utilizing large language models (LLMs) to score the similarity between the generated response and the expected response.

• Explanation generation: To understand the reasoning behind an LLM’s evaluation, SIGNAL IDUNA generates explanations for its judgments. This provides valuable insights into the strengths and weaknesses of the generated responses, helping the developers identify specific areas for improvement.

This multi-stage evaluation approach provides SIGNAL IDUNA a holistic view of the model’s performance, enabling data-driven optimization at each stage. The Looker dashboard plays a vital role in visualizing these metrics, making it easier for the developers to identify areas where the model excels and where it needs improvement.

Real-world impact: AI-powered efficiency and productivity

To determine whether the AI assistant provided measurable added value for its workforce, SIGNAL IDUNA conducted an experiment with a total of 20 employees (internal and with external providers). During the experiment, customer requests were processed with and without the AI knowledge assistant to assess its impact. 

One of the key benefits observed was a reduction in processing time. Searching across numerous data sources used to be a time-consuming process. The experiment showed that using the AI knowledge assistant reduced the core processing time (information search and response formulation) by approximately 30% and increased the quality of the response based on expert evaluations. The time saved was particularly notable for employees with less than two years of experience in health insurance.

In addition, the AI knowledge assistant significantly increased the case closure rate. Health insurance is a very complex field, and the use of external service providers means that not every employee can always answer every customer question. With support from the AI knowledge assistant, SIGNAL IDUNA’s case closure rate increased by approximately 24 percentage points, rising from 73% to almost 98%.

Scaling for the Future

“Together with Google, we at SIGNAL IDUNA have successfully applied gen AI to one of our core business processes” Stefan Lemke, CIO at SIGNAL IDUNA, said. “Now, it’s time to scale this powerful technology across our entire organization. We’re not just scaling a tool, we’re scaling innovation, learning, and the possibilities of what we can achieve.”

Gen AI offers enormous potential for optimizing processes and developing innovative solutions. With its innovative approach — business teams experimenting with the technology in a decentralized manner and developing customized applications — SIGNAL IDUNA is primed to pioneer the next generation of insurance solutions and services. 

At the same time, SIGNAL IDUNA is establishing central standards to scale insights gained across the company and tap into the combined power of its teams, resources, and lines of business. This strategic decision has helped create valuable resources like code libraries, infrastructure blueprints, and centrally offered services. 

By combining agility with established standards and best practices, SIGNAL IDUNA can now react quickly to new requirements, setting a new standard for efficiency and customer satisfaction.

^{This project was delivered by the following core team members, Max Tschochohei, Anant Nawalgaria, and Corinna Ludwig by Google, and Christopher Masch, Michelle Mäding from SIGNAL IDUNA}

Today, we’re sharing the new Gemma 3 model is available on Vertex AI Model Garden, giving you immediate access for fine-tuning and deployment. You can quickly adapt Gemma 3 to your use case using Vertex AI’s pre-built containers and deployment tools. 

In this post, you’ll learn how to fine-tune Gemma 3 on Vertex AI and deploy it as a production-ready endpoint.

Gemma 3 on Vertex AI: PEFT and vLLM deployment

Tuning and deploying large language models can be computationally expensive and time-consuming. That’s why we’re excited to announce Gemma 3 support for Parameter-Efficient Fine-Tuning (PEFT) and optimized deployment using vLLM on Vertex AI Model Garden. 

Gemma 3 fine-tuning allows you to achieve performance gains with significantly less computational resources compared to full fine-tuning. Our vLLM-based deployment is easy-to-use and fast. vLLM’s optimized inference engine maximizes throughput and minimizes latency, ensuring a responsive and scalable endpoint for your Gemma 3 applications on Vertex AI.

Let’s look at how you can fine-tune and deploy your Gemma 3 model on Vertex AI.

Fine-tuning Gemma 3 on Vertex AI

In Vertex AI Model Garden, you can fine-tune and deploy Gemma 3 using PEFT (LoRA) from Hugging Face in only a few steps. Before you run the notebook make sure you complete all of the initial steps as described in the notebook. 

Fine-tuning Gemma 3 on Vertex AI for your use case requires a custom dataset. The recommended format is a JSONL file, where each line is a valid JSON string. Here’s an example inspired by the timdettmers/openassistant-guanaco dataset:

The JSON object has a key text, which should match train_column; The value should be one training data point, i.e. a string. You can upload your dataset to Google Cloud Storage (preferred) or to Hugging Face datasets.

Choose the Gemma 3 variant that best suits your needs. For example, to use the 1B parameter model:

You have the flexibility to customize model parameters and job arguments. Let’s explore some key settings. LoRA (Low-Rank Adaptation) is a PEFT technique that significantly reduces the number of trainable parameters. The following parameters control LoRA’s behavior. lora_rank controls to control dimensionality of the update matrices (smaller rank = fewer parameters), lora_alpha that scales the LoRA updates, and lora_dropout to add regularization. The following settings are a reasonable starting point.

When fine-tuning large language models (LLMs), precision is a key consideration, impacting both memory usage and performance. Lower precision training, such as 4-bit quantization, reduces the memory footprint. However, this can come with a slight performance trade-off compared to higher precisions like 8-bit or float16. The train_precision parameter dictates the numerical precision used during the training process. Choosing the right precision involves balancing resource limitations with desired model accuracy.

Optimizing model performance involves tuning training parameters that impact speed, stability, and capabilities. Essential parameters include per_device_train_batch_size, which determines the batch size per GPU, with larger sizes accelerating training but demanding more memory. gradient_accumulation_steps allows simulating larger batch sizes by accumulating gradients over smaller batches, providing a memory-efficient alternative at the cost of increased training time. The learning_rate dictates the optimization step size, where a rate that is too high can lead to divergence, while a rate that is too low can slow down convergence. The lr_scheduler_type dynamically adjusts the learning rate throughout training, such as through linear decay, fostering better convergence and accuracy. And, the total training duration is defined by either max_steps, specifying the total number of training steps, or num_train_epochs, with max_steps taking precedence if both are specified. Below you have the full training recipe you find in the official notebook.

Finally, create and run the CustomContainerTrainingJob to start the fine-tuning job.

You can monitor the fine-tuning progress using Tensorboard. Once the job is complete, you can upload the tuned model to the Vertex AI Model Registry and deploy it as an endpoint for inference. Let’s dive into deployment next. 

Deploying Gemma 3 on Vertex AI

Deploying Gemma 3 on Vertex AI requires only three steps as described in this notebook. 

First, you need to provision a dedicated endpoint for your Gemma 3 model. This provides a scalable and managed environment for hosting your model. You use the create function to set the endpoint name (display_name), and ensure dedicated resources for your model (dedicated_endpoint_enabled).

Next, register the Gemma 3 model within the Vertex AI Model Registry. Think of the Model Registry as a central hub for managing your models. It keeps track of different versions of your Gemma 3 model (in case you make improvements later), and is the central place from which you’ll deploy.

This step involves a few important configurations including the serving container to deploy Gemma 3. 

To serve Gemma 3 on Vertex AI, use the Vertex AI Model Garden vLLM pre-built Docker image for fast and efficient model serving. The vLLM recipe to set how vLLM will serve Gemma 3 which includes  --tensor-parallel-size lets you spread the model across multiple GPUs if you need extra computation resources, --gpu-memory-utilization controls how much of the GPU memory you want to use and --max-model-len sets the maximum length of text the model can process at once. You also have some advanced settings like --enable-chunked-prefill, and --enable-prefix-caching to optimize performance, especially when dealing with longer pieces of text. 

There are also some of deployment configuration Vertex AI requires to serve the model including the port (8080 in our case) that the serving container will listen on, defines the URL path for making prediction requests (e.g., “/generate”) and the URL path for health checks (e.g., “/ping”), allowing Vertex AI to monitor the model’s status.

Finally, use upload() to take this configuration – the serving container, your model-specific settings, and instructions for how to run the model – and bundle them up into a single, manageable unit within the Vertex AI Model Registry. This makes deployment and version control much easier.

Now you’re ready to deploy the model. To deploy the registered model to the endpoint, use the deploy method as shown below.

This is where you choose the computing power for our deployment including the type of virtual machine (like “a3-highgpu-2g”, machine_type), the kind of accelerator (e.g., “NVIDIA_L4” GPUs, accelerator_type), how many accelerators to use (accelerator_count). 

Deploying the model requires some time and you can monitor the status of the deployment in Cloud Logging. Once you get the endpoint running, you can use the ChatCompletion API to call the model and integrate it within your applications as shown below.

Depending on the Gemma model you deploy, you can use the ChatCompletion API to call the model with multimodal inputs (images). You can find more in the “Deploy Gemma 3 4B, 12B and 27B multimodal models with vLLM on GPU” section of the model card notebook.

What’s next?

Visit the Gemma 3 model card on Vertex AI Model Garden to get started today. For a deeper understanding of the model’s architecture and performance, check out this developer guide on Gemma 3.