gcp

Editor’s note: Virgin Media O2 provides internet and communications services to more than 48.5 million subscribers, and teams are also responsible for supporting more than 16,000 employees. Virgin Media O2 is committed to empowering customers with outstanding customer-centered products and essential connectivity — all powered behind the scenes by Google Cloud, the backbone of the company’s daily operations. In this guest blog, Henry Tze shares his DevOps insights.

From virtual machines to applications to storage assets, at Virgin Media O2 we have a huge cloud footprint, all hosted in Google Cloud. Google Cloud enables us to process vast amounts of data at scale and deliver cutting-edge customer services, using an array of cloud-first services, including BigQuery, Google Kubernetes Engine, and Compute Engine. 

All told, we’re managing more than 11 million resources, including nearly 1 million BigQuery tables. Ensuring adherence to stringent security and compliance requirements is central to our mission — and no small feat.

Reducing identity sprawl to safeguard our cloud environment

We have a large, international team, and even though all of our employees are vital to our operations, they don’t all need access to the same information. To protect both our internal and customer data, we have to carefully configure our Identity and Access Management (IAM) infrastructure to ensure we’re compliant with the U.K.’s stringent data management regulations.

Part of our security journey has involved both our migration to the cloud and fully embracing Infrastructure as Code (IaC). We now refer to our infrastructure as “everything as code,” using GitOps to centralize our source of truth. Doing so helps us create a clear audit trail and version history record. 

This centralization has helped us align developers, engineers, and compliance teams toward the same goals. Unfortunately, ensuring that these teams have access to the tools they need to accomplish those goals is complicated.

Since taking a GitOps approach means we’re focusing more on what we want our infrastructure to look like than how we get there, we have less insight into the specific tools or paths our teams need to take to get the job done. However, this led to permission sprawl as users gained access to applications or datasets for a specific purpose and retained that access after they were finished with their work.

As that sprawl expanded, it was difficult to track who had access to what which ultimately posed a challenge to our compliance and security goals. Balancing ease of access with security compliance is a complicated challenge, but we needed to find a simple solution. 

With Google Cloud, we already had access to IAM conditions, which helped us customize time-based access controls. However, for us to scale we needed to automate provisioning and permissions management based on roles and responsibilities, and now we can with Google Cloud’s built-in Privileged Access Manager (PAM). 

Improving compliance by centralizing and automating identity controls with PAM

Using PAM, our team can ensure we adhere to least-privilege standards to protect our assets but still automate access for appropriate users, all without needing an external agent or additional software tool. By providing just-in-time (JIT), time-bound, and approval-based access, our admin team can rest easy knowing that the only users who are actively using an application or other asset have access. 

Our administrators can review access permissions in a single place and still customize permission controls from asset to asset. For example, developers may have predefined, role-based permissions for their day-to-day applications. However, if they’re assigned an ad hoc data project, they’ll need to request temporary access, and the approver is notified immediately. 

This extra level of caution helps us mitigate risk by reducing the number of users who have blanket permissions and prevents access sprawl as we grow. Our compliance auditors are also satisfied because we have clear access permission histories which simplifies and centralizes the audit process. 

Using Google Cloud’s PAM we created a self-service model at Virgin Media O2, that has helped us streamline access, boost productivity, and address compliance. Here are core tenets of our model:

Delegated approvals: Our Cloud Platform Security Team can delegate approval authority to Department and Product Administrators, which can enable them to manage privileges for their areas of expertise.

Built-in guardrails: Our Cloud Platform Security Team relies on predefined templates with embedded security controls, which can ensure that privileged access is granted securely, even with delegated approvals.

Self-sufficient teams: Our Department and Product administrators gain the autonomy to manage entitlements independently, so they can make changes swiftly and without the need to involve the security team.

On-demand access: Our customers can request permissions at any time, and adapt to evolving work requirements.

Compliance audits: Our Compliance Auditors have complete visibility into every action related to privileged access, from entitlement creation and grant requests to activation and termination.

Using PAM to drive success

Since adopting PAM, we’ve successfully reduced longstanding IAM permissions that created potential vulnerabilities in our cloud and simplified access management across the organization. Most importantly, PAM has helped us achieve and maintain compliance with the stringent requirements imposed on our regulated organization. 

We’ve also built a more sustainable and scalable access system that will grow alongside Virgin Media O2 as we continue to supercharge more customers across the UK.

One of the biggest areas of promise for generative AI is coding assistance — leveraging the power of large language models to help developers create or update application code with amazing speed and accuracy, dramatically boosting productivity. For Gemini Code Assist, 2024 has been a year of tremendous growth and innovation, as we help companies build, run, and operate apps across the software development lifecycle (SDLC). Just last week, we announced support for Gemini 2.0 Flash. This new large language model (LLM) will help Gemini Code Assist users be more productive with higher quality responses and lower latency so they can stay in an uninterrupted flow state for longer. 

Gemini Code Assist tools: A connected development ecosystem

Building on these advancements, today we’re launching Gemini Code Assist tools, a new way for developers to get the information they need to build and manage apps. As of today, you can sign up to preview Gemini Code Assist tools and start testing them. Or, sign up now to build your own extension for Gemini Code Assist.

Tools enable developers to retrieve information from, or act on any part of their engineering system, which is especially helpful for services outside the IDE. For example, you might summarize recent comments from a Jira issue, find the last person who merged changes to a file in git, or show the most recent live site issue from Sentry. Tools can do this because they leverage that most basic building block of the internet: the API. Tools effectively translate any natural language command into a parametrized API call as defined by the OpenAPI standard or a YAML file you provide.

Here are a few ways Gemini Code Assist tools can enhance your development experience:

• Uninterrupted flow: Accessing crucial information and tools from applications without ever leaving your IDE. This minimizes distractions and maximizes focus, boosting your productivity and code quality.

• Data-driven decisions: With real-time access to data and insights from integrated partner tools, you can make informed decisions faster and more consistently. This leads to more efficient development cycles, reduced errors, and ultimately, better software.

• A comprehensive toolkit: We are constantly expanding our ecosystem by onboarding new partners and open-source technologies, creating a rich and diverse toolkit within Code Assist, ensuring you have everything you need to succeed at every stage of development.

Getting scalable, secure applications into production requires more than just writing great code — developers need solutions for productivity, observability, security, databases, and more. We’re excited to partner with industry leaders who are eager to improve the developer experience. 

Starting today, in our private preview, you can use the new tools feature in Gemini Code Assist from our launch partners including Atlassian (Rovo), GitHub, GitLab, Google Docs, Sentry, and Snyk to provide the first look into how your software development experience is better when you have information available right in the IDE.

“Gemini Code Assist is a major step forward in keeping developers in the flow by reducing the need to context switch. By connecting Atlassian Rovo to Gemini Code Assist, engineering organizations will be able to leverage the rich context already captured in tools like Jira and Confluence — where teams ideate, plan, and document technical requirements. This integration ensures every developer can instantly access technical specifications, tasks in progress, blockers, or even identify the right person to ask for help, all without leaving their coding environment.” – Josh Devenny, Head of Product for Agile and DevOps AI, Atlassian

“By leveraging GitLab’s open API, Google has seamlessly integrated GitLab with Gemini Code Assist, empowering joint customers to boost efficiency and reduce context-switching by accessing GitLab functionality directly within their IDE. This integration offers another flexible and seamless way to leverage GitLab’s industry-leading DevSecOps platform to streamline workflows and accelerate secure software delivery.” – Emilio Salvador, GitLab, VP, Strategy and Developer Relations, GitLab

“We’re thrilled to partner with Google Cloud on this initiative. Integrating Sentry’s error monitoring and performance insights directly into the developer workflow through Gemini Code Assist tools empowers developers to build and ship higher-quality software faster than ever before.” – Dave Rosenthal, CTO, Sentry

“The transformative potential of generative AI is reshaping how developers create software, and at Snyk, we’re committed to ensuring this innovation happens securely. By collaborating with Google Cloud to build a dedicated Gemini tool, we’re empowering developers to use Gemini to write code and Snyk to test and fix it — all within their IDE. This integration ensures developers can stay in their creative flow, leveraging AI to accelerate innovation while maintaining the highest standards of security.” – Danny Allan, Chief Technology Officer, Snyk

A broad ecosystem of partners

Recognizing the diverse tools developers use, we’re collaborating with many partners to integrate their technologies directly into Gemini Code Assist for a more comprehensive and streamlined development experience. These partners, and more, help developers stay in their coding flow while accessing information through tools that enhance the SDLC.

Observability

“Dynatrace is thrilled to be a launch partner for Gemini Code Assist. This collaboration with Google Cloud will further empower our customers to harness the full potential of AI, enhancing their coding efficiency and accelerating innovation. Together with Google Cloud, we’re committed to helping organizations achieve greater resilience and stay ahead in a rapidly evolving digital landscape.” – Raj Ramanujam, Regional Vice President of Global Alliances, Dynatrace

“We are excited to partner with Google Cloud and join the Tools Partner Ecosystem for Gemini Code Assist. This collaboration deepens our shared commitment to improving developer productivity — a core value at Harness — by delivering innovative AI-native solutions to accelerate and enhance software delivery.” – Brad Rydzewski, Vice President of Engineering, Harness

“Switching tools and contexts is a barrier to productivity and can result in software outages that impact business. The New Relic Intelligent Observability platform is uniquely built to enable agentic orchestration, which allows us to directly integrate with partners like Google to deliver real-time, understandable observability insights and intelligent recommendations wherever practitioners want them. Together, Google and New Relic enable anyone supporting software to act faster and smarter to prevent business-impacting incidents.” – Manav Khurana, Chief Product Officer, New Relic

Security in the SDLC

“AI is transforming the way developers work, streamlining processes and reducing the toil associated with writing code. We are excited to partner with Google on the Gemini Code Assist tools program to further advance the adoption of AI in software development. As AI becomes more integrated into the SDLC, maintaining trust in your codebase requires new strategies and tools. Through this partnership, Gemini Code Assist and SonarQube work in tandem to help developers validate AI-generated code at scale while leaders can maintain the confidence that their code remains secure, reliable, and trustworthy.” – Harry Wang, VP of Growth & New Ventures, Sonar (SonarQube)

“The opportunity to join the Gemini Code Assist tools ecosystem allows Black Duck to broaden how we empower developers to build secure applications with greater speed and confidence. By integrating our leading application security solutions with Google Cloud’s advanced AI, developers can proactively identify and mitigate vulnerabilities within their code, ensuring the integrity of their software supply chain while streamlining development workflows.” – Scott Johnson, Vice President of Product Management, Black Duck

Databases

“Aiven is thrilled to join the Gemini Code Assist Partner Ecosystem. We believe the collaboration between Google Cloud and Aiven will empower customers with seamless access to Aiven’s fully managed data platform directly within their coding environment. This integration will unlock new levels of productivity and efficiency for developers building cutting-edge applications on Google Cloud.” – John Kennedy, Head of Databases, Aiven

”We’re excited to bring together DataStax Astra DB, Langflow, and Google Gemini Code Assist. This integration empowers developers to seamlessly access their database structures, create data models, get query assistance, and design flows — all within their IDE. By embedding these capabilities directly into the coding environment, we’re significantly streamlining development workflows and enhancing productivity for developers who rely on Astra DB and Langflow.” – Ed Anuff, CPO, DataStax

“We look forward to exploring the use of Google Code Assist tools with Elastic AI Assistant. We are optimistic about the potential this integration has in optimizing development workflows, making it easier for our joint customers to build intelligent search experiences, proactively address performance issues, and strengthen application security.” – Steve Kearns, General Manager of Search, Elastic

“Gemini Code Assist is a powerful tool that helps customers and partners build applications faster. MongoDB is proud to be a launch partner with Google Cloud on the Gemini Code Assist tools ecosystem, to significantly boost development productivity at the most crucial layer of their application — the data layer. This integration will supercharge Gemini Code Assist with MongoDB domain knowledge, allowing customers to connect and interact with their MongoDB clusters using natural language.” – Andrew Davidson, SVP of Product, MongoDB

“Neo4j customers currently use Gemini to automate the creation of knowledge graphs and then to query those graphs with a GraphRAG pattern. The Neo4j integration with Gemini Code Assist tools  will simplify this architecture, reducing the cost and time for our customers to get value from these systems. Graphs allow our customers to get value from the connections in their data. We see application across verticals, including improving fraud ring detection, optimizing supply chains and improving drug discovery.” – Sudhir Hasbe, Chief Product Officer, Neo4j

“Redis is known to be loved by developers as we always strived for the best developer experience. We’re excited to partner with Google to create the Gemini Code Assist tool for Redis that enables developers to seamlessly query and validate data, as well as generate code that follows best practices and avoids anti-patterns, all without disrupting their development flow.” – Pieter Cailliau, Head of Developer Experience, Redis

“A seamless developer experience means staying in the flow without interruptions or context switches. Today, this is more achievable than ever, thanks to gen AI-driven query generation and code completion that integrate database development directly into the coding environment. With Gemini Code Assist, developers can now handle all their database tasks within the same interface they use to write code, supercharging their productivity and making the entire coding experience up to 10X faster and more efficient. We’re thrilled to empower developers to work smarter and faster with this all-in-one solution!” – Rahul Rastogi, Chief Innovation Officer, SingleStore

Partnering with the developer and application tools ecosystem

Bringing tools for Gemini Code Assist is not just beneficial for developers, it also provides immense value to our partners, offering exciting new opportunities for growth and engagement. This new feature will help Google Cloud’s partners:

• Foster direct developer connections: Bring the power of your tools directly into the hands of developers, providing unparalleled access and visibility within their daily workflow.

• Expand reach: Tap into the vast and growing user base of Gemini Code Assist, extending your reach and brand awareness within the developer community. We also proactively co-market with our partners to maximize exposure and tell a cohesive story.

Code Assist currently provides developers with a natural language interface to both traditional APIs and AI Agent APIs. Partners can quickly and easily integrate to Code Assist by onboarding to our partner program. The onboarding process is as simple as providing an OpenAPI schema, a Tool config definition file, and a set of quality evals prompts used to validate and tune the integration. Code Assist currently supports authentication to partner APIs via OAuth 2.0 Authorization Code grant type, also known as a 3LO (3-Legged OAuth) flow. In the future, we plan to add support for API Key auth as well. 

For more information or to request access to the partner program, please fill out the signup form.

Join the future of software development

The future of coding is collaborative, connected, and AI-powered. If you’re a developer who is ready to experience the power of connected development, please sign up to access the private preview of Gemini Code Assist tools. Developers can also sign up to use Gemini 2.0 Flash in Gemini Code Assist, for enhanced coding assistance capabilities in popular IDEs such as Visual Studio Code, IntelliJ, PyCharm and more. And if you are interested in building an extension for Gemini Code Assist, please use our signup form to get in contact with us.

Looker, Google Cloud’s complete AI for BI platform, is now available on the AWS Marketplace, allowing AWS customers to benefit from Looker’s powerful analytics and reporting capabilities in their environment. As the next step in Looker’s commitment to an open multicloud strategy, this milestone simplifies procurement for businesses that use AWS cloud and databases such as Amazon Redshift, empowering them to quickly and easily integrate Looker’s rich AI-driven visualizations and insights into their existing infrastructure.

Today’s businesses leverage data as a competitive advantage — all the more so with the accelerating adoption of generative AI. To unlock the full potential of your data, you need a business intelligence platform that’s:

• Built for self-service, so all employees can access and benefit from data

• Delivers instantaneous AI-powered insights to any part of the organization

• Backed by a semantic layer, helping ensure that decisions are based on the foundational truth underpinning your business

Looker is the engine that enables organizations to imagine new AI-powered capabilities and showcase their data insights in a highly visual and easily understandable way. With Gemini in Looker, including Conversational Analytics, you can have a natural language conversation with your data, and produce rich data visualizations with Studio in Looker. This complete AI-for-BI solution lets teams connect to multiple diverse data sources and create a unified view of their information, uncovering trends and embedding analytic-driven decision-making in applications and throughout the enterprise.

Your employees rely on the data and tools they use every day to be accurate and work as anticipated. With this update, Looker brings its pioneering semantic layer to AWS environments, delivering consistent, governed and trusted data definitions to the organization. This allows AI models to work against high-quality data, ensuring its insights are accurate and trustworthy.

Looker on the AWS Marketplace delivers:

• Multicloud flexibility: Looker lets organizations leverage powerful AI-for-BI capabilities across multiple clouds, including AWS, helping them gain insights within their preferred environment, avoid vendor lock-in, and easily adapt to evolving cloud strategies as needs change.

• Simplified procurement: You can streamline procurement and consolidate billing, maximizing AWS investments by purchasing Looker directly through the AWS Marketplace. This simplifies cloud investments so you can increase the value of your AWS environment.

• Universal semantic layer for trustworthy data: Looker for the AWS Marketplace customers can now build a reliable, consistent foundation for AI-driven insights. Standardizing data definitions and governance across platforms minimizes data silos and enhances data accessibility, helping organizations to make smarter, accurate, data-driven decisions.

• Access to Google AI-powered BI on AWS: Harness the power of Google Cloud’s advanced AI capabilities within your existing AWS environment, including powerful machine learning, predictive analytics, and AI-driven insights including through natural language interactions driven by Conversational Analytics.

Looker is the BI engine for your AI-driven future. To experience the power of this multi-cloud BI solution, start with Looker on AWS Marketplace today, or learn more about Looker at cloud.google.com/looker.

AI is rapidly transforming industries and redefining the future of work. However, many organizations face a significant hurdle: bridging the knowledge gap and acquiring the necessary skills to effectively harness the power of AI. 

Recognizing this challenge, Google Cloud is set to launch the AI Playground in Shoreditch, Central London, in the first quarter of 2025. This innovative space will serve as a dynamic hub for businesses and individuals to demystify AI, explore its potential, and develop practical expertise.

More than a showcase — dive deep into AI

Google’s powerful Gemini model family — now in its supercharged second generation — takes center stage at the AI Playground, featuring several interactive demos that put its multimodal and agentic capabilities on display. Among the capabilities guests can experience and experiment with first hand are Gemini’s ability to analyze complex data, generate creative formats, and power innovative solutions.

The AI Playground is much more than a technology showcase. 

We’ve built the space to serve as an immersive learning environment where visitors can actively engage with AI, participate in hands-on workshops and hackathons, and connect with Google Cloud AI experts. This unique approach fosters a deeper understanding of AI concepts and encourages experimentation with cutting-edge tools and techniques.

Addressing the AI skills gap

The AI Playground directly addresses the growing need for AI skills development in today’s rapidly evolving technological landscape, providing a dedicated space for:

• Hands-on experimentation: Gain practical experience with cutting-edge AI tools and techniques, moving beyond theoretical knowledge to real-world application.

• Skills development: Build on-demand AI skills through interactive workshops and hackathons led by Google Cloud experts, equipping individuals and teams with the expertise needed to thrive in the AI era.

• Community building: Connect with fellow AI enthusiasts, share knowledge, and facilitate collaboration, creating a vibrant ecosystem for learning and innovation.

• Real-world inspiration: Explore AI applications across diverse industries and discover new possibilities for your organization, sparking creativity and driving the development of novel AI solutions.

Empowering the future of AI

By providing accessible learning opportunities and fostering a thriving AI community, the AI Playground empowers individuals and businesses to embrace the transformative power of AI and contribute to shaping a better future. It’s a place where curiosity meets innovation, learning is hands-on, and  the potential of AI is unlocked. Mark your calendars for Q1 2025 and prepare to embark on your AI journey at Google Cloud’s AI Playground.

Enterprise workloads like SAP S/4HANA present unique challenges when migrating to a public cloud, making the choice of a cloud provider critically important. As an in-memory database for large SAP deployments, SAP HANA can have massive memory and CPU processing requirements, and that can just be the beginning of the challenges. Businesses are generating, processing, and making business decisions on more data than ever before, and they require increasingly complex data analysis, algorithms, and reporting. SAP provides the real-time operational and historical analytics and insights businesses need to make informed decisions quickly. However, this requires machines that can process and analyze large volumes of data in real time, demanding high levels of memory and compute power. 

To meet these demands, we recently introduced X4 machine types. The Compute Engine X4 machine family is purpose-built to handle the demanding requirements of SAP HANA Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) workloads. These machines deliver strong performance, scalability, and reliability, empowering businesses to unlock the full potential of their SAP S/4HANA, SAP Business Suite (ECC) on SAP HANA deployments, and SAP’s Industry Solutions. X4 is also built to support OLAP workloads such as SAP BW/4HANA and SAP BW on HANA.

Uncompromising performance in a scalable solution

As businesses grow, whether organically or via acquisitions, they often risk hitting CPU and memory limitations of their SAP HANA system. This forces them to upgrade to larger scale-up machines earlier or using multiple machines in a scale-out configuration. Scaling out is a complex custom process that requires application data redesign and approvals from SAP, as well as being costly from an operational and resource perspective. In addition, the planned downtime required for these migration operations can be unacceptable for organizations that rely on SAP for mission-critical operations.

X4 solves these challenges by allowing businesses to grow further in scale-up mode before hitting performance bottlenecks. X4 is available in 16TB, 24TB, and 32TB memory configurations and 960, 1440, 1920 vCPU cores respectively with “standard sizing” SAP certification for both SAP HANA OLTP and OLAP capable of running the most demanding enterprise SAP HANA workloads. X4 machines also boast scale-out OLTP certifications for up to four nodes the size of the servers, for a total of 128 terabytes for S/4HANA use cases — larger than any other cloud provider’s. The X4 16TB machines achieved an SAP Benchmark IaaS SAPS result that is over 8% higher than the closest IaaS cloud’s. (Source: SAP note 2456432 – Logon required). 

As the only cloud IaaS provider offering a 32TB SAP-certified machine, Google Cloud helps ensure that your SAP HANA environment grows seamlessly with your business needs while providing a cost-effective and simple cloud-native experience. Here’s how:

• Industry-leading compute and storage: The Google Cloud X4 machine family gains its power and scale through a combination of powerful hardware and software optimizations. Integration with Google Cloud Hyperdisk provides high performance, scalability, and resilient block storage. Hyperdisk provides dedicated, dynamically tunable read/write operations per second (IOPS) and throughput with each volume up to 5GB per second, delivering substantially higher performance compared to the previous generation of Persistent Disk. This level of performance is crucial for applications that require rapid data access, like databases, real-time analytics, and reduces SAP HANA rehydration times. Additionally, X4 machines run on one of the world’s largest privately managed networks, offering low-latency performance. 

• High availability and reliability: Minimizing downtime for mission-critical SAP applications is a top priority for customers. X4 machines have an industry-leading 99.95% Compute Engine Memory-Optimized Single Instance SLA. Paired with Hyperdisk, X4 enables rapid SAP HANA rehydration capabilities (<45 min full rehydration for 32TB) that minimize application business process downtime, so your mission-critical SAP applications stay up and running. 

• Performance-boosting innovation: Titanium is a hardware- and software-based technology designed to significantly enhance workload performance, efficiency, and security while also simplifying management by automating many infrastructure-level tasks. Titanium shifts networking and storage functions away from the machine’s CPU to specialized hardware, allowing the machine’s CPU to focus on running SAP HANA applications. Titanium’s tight integration with Google Cloud’s Hyperdisk block storage service allows for best-in-class storage I/O performance (up to 400k IOPS per instance), surpassing what’s typically possible with traditional architectures. 

• Enhanced security and compliance: X4 machines inherit Google Cloud’s robust security features, including encryption of data in transit and at rest as well as compliance with industry standards and regulations with support for Google Cloud’s Assured Workloads. Titanium enhances security feature performance by offloading security-sensitive tasks like encryption and key management to dedicated hardware with enhanced protections. This helps improve the security posture of workloads, reduces the risk of vulnerabilities, and enables customers to meet their compliance and regulatory requirements.

• Simplified cloud-native experience: X4 machines offer the benefits of a cloud-native architecture for SAP S/4 HANA, integrating seamlessly with Google Cloud’s ecosystem and providing access to a wide range of cloud services. Compute Engine also offers a seamless migration experience for X4. You can easily migrate your SAP instance from M2 servers (up to 12 terabytes) to X4. The migration process from other servers or on-prem environments to X4 is the same as the standard process for migrating to any other Google Cloud server. 

• A choice of service model: When migrating to Google Cloud, SAP customers have two choices. You can choose a direct (BYOL) deployment model, or you can take advantage of RISE with SAP, which brings solutions and expertise from SAP and technology ecosystem partners into one single-contract subscription offering.

  “In the past few years, our SAP HANA systems have seen significant data growth with an increasing need for higher performance. With the 24TB X4 machines and Hyperdisk storage, we have been able to raise the ceiling for our future data growth and are also looking to see improvements in our performance. Added to this, Google’s X4 machines are cloud native, giving us opportunities to automate system management and operations.” – Shawn Lund, US Chief Technology Officer, Deloitte

Google Cloud’s X4 machines represent a significant advancement in SAP HANA infrastructure, offering high performance, scalability, and cost-efficiency. By integrating with Google Cloud’s powerful ecosystem and providing a cloud-native experience, X4 machines empower businesses to unlock the full potential of their SAP S/4HANA deployments and drive digital transformation. 

Learn more about X4 machines and RISE with SAP on Google Cloud. View SAP Datasphere for Google BigQuery on Google Cloud Marketplace.

As cybersecurity threats have grown more sophisticated and prevalent, we’ve seen organizations develop robust cyber threat intelligence (CTI) programs to help bolster defenses. However, creating and maturing a CTI program remains a challenge because it requires people, processes, technologies, and metrics to validate success.

To help organizations better operationalize threat intelligence, we’ve published the CTI Program Design Playbook. This training track is the inaugural release from Mandiant Academy’s newest on-demand training, developed for professionals who actively defend networks. It’s designed to provide you with the essential knowledge and skills to design, build, operate, and optimize a CTI program from the ground up.

What you’ll learn: An overview

This track includes three two-hour courses addressing different aspects of CTI program development.

In the first course, Designing and Building a CTI Program, you will learn how to establish the foundations of a CTI program, including a CTI program charter and roadmap.

In the second course, Operating an Effective CTI Program, you will learn how to effectively identify and maintain a CTI program’s stakeholders. This course also describes the resources necessary to collect, analyze, and disseminate threat intelligence while accommodating stakeholders’ unique requirements.

Students who are part of an active CTI program but would like to mature their team’s capabilities may benefit from the third course, Optimizing an Efficient CTI Program. This course teaches you helpful ways to accurately measure a CTI program’s effectiveness and, where necessary, make improvements.

Who should take these courses?

This training track is the inaugural release from Mandiant Academy’s newest approach to on-demand training, which breaks the core tenets of effective cybersecurity strategies into discrete two-hour courses. This new style aims to address the needs of busy practitioners who already possess the knowledge and skills needed to defend networks but have limited time to enhance their capabilities. 

The CTI Program Design Playbook was developed for practitioners:

• Intelligence analysts supporting network defenders at the tactical, operational, or strategic levels

• Managers of active intelligence programs who would like to improve their team’s capabilities

• Cybersecurity practitioners who desire an intelligence capability

• Practitioners tasked with founding their organization’s first intelligence capability

Start learning today

To learn more about the CTI Program Design Playbook and to sign up for the course, please visit our website. You can access a wealth of knowledge through Mandiant Academy’s on-demand, instructor-led, and experiential training options. We hope this course proves helpful in your efforts to defend your organization against cyber threats.

As a Kubernetes platform engineer, you’ve probably followed the buzz around eBPF and its revolutionary impact on Kubernetes networking. Perhaps you’ve explored Cilium, a popular solution leveraging eBPF, and wondered how Google Kubernetes Engine (GKE) harnesses this power. That curiosity may have led you to GKE Dataplane V2. And you’ve undoubtedly noticed that where eBPF goes, enhanced observability follows.

In this article, we’ll shine a light on a tool that you can leverage with GKE Dataplane V2: Hubble. This powerful observability tool provides deep visibility with added Kubernetes context into your network packet flow.

GKE Dataplane V2 observability provides a Managed Hubble solution. This includes the following components:

• Hubble Relay: a service that collects network telemetry data about your pods and nodes

• Hubble CLI: a command-line interface tool providing live traffic information within the cluster

Enabling Dataplane V2 observability in GKE, either through the Google Cloud console or the gcloud command, triggers the deployment of a dedicated hubble-relay deployment with a single pod. This pod houses both the Hubble Relay and CLI components as dedicated containers, providing immediate access to network telemetry data and command-line tools for analysis.

While the Hubble UI is not included in the initial deployment, you can easily enhance your observability setup by adding it. This web-based interface offers a visual and interactive way to explore and analyze the data collected by Hubble Relay, making it easier to identify trends, spot anomalies, and troubleshoot issues.

With Hubble CLI and UI at our disposal, let’s explore how these tools can be used to effectively troubleshoot your Kubernetes environment.

Before you start

• Make sure you’re comfortable using the Hubble CLI (link to documentation).

• Know how to deploy the Hubble UI binary distribution (link to documentation).

Setting the stage

For this example, we’ll use two pods, each deployed in a separate namespace and exposed as a service.

Scenario 1: Tracing a simple request

In this scenario, we observe a basic interaction between a pod and a service.

1. Initiate a request: Execute a curl request from the store-frontend pod to backend-svc service:

$ kubectl exec store-frontend -n fe -it -- curl “backend-svc.be.svc.cluster.local.”

2. Observe with Hubble: Use the Hubble CLI to examine the traffic:

$ hubble observe --pod fe/store-frontend --follow

Hint: For a full list of Hubble filters and flags, run hubble observe –help

Hubble captures the entire flow: the store-frontend pod contacting kube-dns for service name resolution, followed by the TCP connection to the store-backend pod. See how Hubble adds the namespace and pod names to the packets? This Kubernetes-aware insight makes it much easier to understand and analyze your network traffic.

Scenario 2: When DNS goes wrong

Let’s simulate a scenario where our DNS server is overwhelmed. We deploy an application that floods the cluster with DNS requests, putting significant strain on the kube-dns pods. Then, we replicate the request from Scenario 1.

By examining the timestamps of the DNS request and response in the Hubble CLI output, we can clearly observe a delay of approximately 5 seconds for kube-dns to respond. This highlights the impact of the overloaded DNS server and explains the latency observed in our system.

Let’s intensify the load on our DNS server even further, pushing it to the point where DNS requests fail outright due to an inability to resolve service names.

$ kubectl exec -it store-frontend -n fe -- curl "backend-svc.be.svc.cluster.local."
curl: (6) Could not resolve host: backend-svc.be.svc.cluster.local.

In this extreme scenario, the Hubble drop reveals that our DNS queries are simply not being answered, indicating a complete breakdown in name resolution.

We finally will make the kube-dns pods unavailable by taking them down.

Hubble’s flow logs provide a clear picture of what’s happening. You can see that the store-frontend pod attempts to reach the kube-dns service, but the request fails because there are no healthy kube-dns pods to handle it.

Hubble CLI isn’t just a troubleshooting tool; it’s also powerful for optimization. For example, while running the first scenario, you may have noticed a surprisingly high number of DNS requests generated from a single curl command:

A quick investigation reveals the root cause: a missing trailing dot in the service name, and the pod’s resolv.conf being configured with ndots:5. This combination meant the query wasn’t treated as absolute and was instead expanded with all five search domains listed in resolv.conf, resulting in six DNS requests for every single curl call.

Scenario 3: Network policy mishap!

It appears there was a slight oversight with a new Ingress NetworkPolicy. Let’s just say Monday mornings and network policies don’t always mix…! The front-end application was accidentally left out of the allowlist.

Communications between the front-end and the back-end applications hang and timeout. The front-end engineer rushes to Hubble CLI. Hubble shows that the front-end application traffic is being denied and dropped by a network policy.

Thankfully, the Hubble CLI allows you to filter events by type using the –type flag. For instance, --type policy-verdict shows only events related to network policies. You can further refine this by using --verdict DROPPED to see only events where traffic was dropped due to a policy.

Hubble UI

While the Hubble CLI is undeniably powerful, the Hubble UI offers a complementary and equally valuable perspective. Beyond replicating the CLI’s functionality, the UI provides a visual map of your cluster’s network traffic. This allows you to easily grasp how pods and Services connect, both internally and to the outside world. With filtering options like namespaces and labels, you can quickly isolate specific traffic flows and troubleshoot potential problems.

Explore the depths of GKE Dataplane V2 with Hubble

This exploration of Hubble, while introductory, provides a solid foundation for understanding its capabilities. Remember that far more intricate queries and filters can be crafted to gain deeper insights into specific network issues. Simply knowing about Hubble and how to access it is a significant first step towards effectively operating and troubleshooting Cilium-based Dataplane V2 clusters. As you dive deeper into Hubble’s capabilities, you’ll discover its full potential for observability and optimization within your GKE environment.

When it comes to AI, large language models (LLMs) and machine learning (ML) are taking entire industries to the next level. But with larger models and datasets, developers need distributed environments that span multiple AI accelerators (e.g. GPUs and TPUs) across multiple compute hosts to train their models efficiently. This can lead to orchestration, resource management, and scalability challenges.

We’re here to help. At Google Cloud, we provide a robust suite of GPU and TPU resources alongside advanced orchestration tools as part of AI Hypercomputer architecture to simplify distributed, large-scale training. In this blog, we’ll guide you through the orchestration tools available for GPU accelerators on Google Cloud that can help you streamline and scale your machine learning workflows.

Choose the right accelerator family

A key element of distributed training lies in selecting the right GPU. Google Cloud’s specialized machine families offer tailored solutions for varying needs of performance and cost efficiency. The A3 machine series, featuring NVIDIA H100 and NVIDIA H200 (upcoming) GPUs, delivers strong GPU-to-GPU bandwidth that’s a great fit for large-scale training workloads. In contrast, the A2 machine series with NVIDIA A100 GPUs is designed for scenarios that require minimal inter-node communication such as streamlined, single-node training. Additionally, the versatile G2 machine family, equipped with NVIDIA L4 GPUs, provides the flexibility necessary for inference and testing workloads.

We also offer multiple GPU consumption models to meet the needs of large-scale training:

• Committed Use Discounts (CUDs) provide significant cost savings and guaranteed capacity in return for a long-term commitment.

• Dynamic Workload Scheduler (DWS) comes in two modes, which are designed for various workloads that need assurance or can be flexible about start time; the capacity is available for a defined duration and offered at a lower list price.

• On-demand consumption is the most flexible, with no upfront commitments, although the capacity availability is not guaranteed.

• Spot VMs provide drastically lower costs but are preemptible, requiring resilient and disruption-tolerant job designs.

To further accelerate your distributed training, we’ll explore three powerful orchestration strategies on Google Cloud: Google Kubernetes Engine (GKE), Cluster Toolkit, and Vertex AI custom training pipeline. Each approach brings its unique strengths, enabling you to leverage Google Cloud’s powerful infrastructure to drive your machine learning projects forward quickly and scalably.

Let’s walk through each of the options to better understand how Google Cloud’s advanced orchestration tools can help you optimize resources, reduce complexity, and achieve strong performance in your ML initiatives.

Option 1: GKE for unified workload management

Enterprises with robust platform teams often want a unified environment on which to run all their workloads, including custom training, for simpler management. GKE is a good choice in this context, providing the flexibility and scalability to handle diverse workloads on a single platform. With GKE, platform teams gain centralized control and visibility, while optimizing resource utilization and streamlining management.

Here’s how to orchestrate ML workloads running on GKE:

1. GKE cluster and nodepool provisioning
If you have reservation (CUD or DWS calendar) and prefer to use Terraform, follow the instructions from cluster provisioning templates, and specify the parameter file (terraform.tfvars):

Then execute the following command to provision the GKE cluster and nodepool:

In addition, Cluster Toolkit includes terraform based example blueprints to provision A3 or A3 Mega GKE clusters and nodepool. 

If you prefer to use the gcloud command, follow the step-by-step instructions from this tutorial to create a GKE cluster and nodepool with A3/A3 Mega VMs. 

For DWS Flex, you can create a DWS enabled node-pool with these gcloud commands:

2. Enable GPU direct communication with A3 TCPX/A3 Mega TCPXO and perform an initial benchmark test
Follow these steps to install GPUDirect for TCPX/TCPXO libraries, configure NCCL, and deploy a test workload to perform your initial benchmark tests.

Validate the output of allgather benchmark tests for two A3 Mega nodes:

In the above benchmark output table, the first column is message size, while the algbw and busbw columns on the right indicate per GPU bandwidth. Usually, we use the in/out-of-place busbw column with the biggest message size (highlighted row) to determine cross-node bandwidth. For A3 Mega nodes, we expect a range of 185-190GB/s per GPU; this may indicate near cross-node 1600gbps network bandwidth for A3 Mega nodes with 8 NVIDIA H100 GPUs and 8 NICs.

You may expand the NCCL tests from two nodes to 8, 16, 32, etc. to ensure your cross-node network performance is within a decent range and that all the nodes are healthy.

3. Configure distributed training batch workload
You can use JobSet, a Kubernetes-native API for managing a group of k8s Jobs as a unit using a unified API, to deploy distributed HPC (e.g., MPI) and AI/ML training workloads (PyTorch, Jax, Tensorflow etc.) on Kubernetes.

The following example illustrates a JobSet yaml manifest for A3 with GPUDirect-TCPX, which includes:

1. Key JobSet configuration elements

b. Training job settings, including pytorch main container
c. gcsfuse, tcpx (A3 high), tcpxo (A3 Mega) RxDM container
d. NCCL environment variables

For DWS batch workloads, please refer to the following A3 Mega-based example, integrating Kueue and JobSet settings.

Lastly, refer to this Helmchart example to see how to perform Megatron LM (Llama2) training on A3 Mega.

Option 2: Slurm via Cluster Toolkit

Slurm is one of the most popular high-performance computing (HPC) job schedulers. Used by researchers in both academia and industry, it offers a robust solution for LLM training orchestration with familiar semantics. Support for Slurm on Google Cloud is provided by Cluster Toolkit, formerly known as Cloud HPC Toolkit, open-source software that simplifies the process of deploying HPC, AI, and ML workloads on Google Cloud. It is designed to be highly customizable and extensible, and to address the deployment needs of a broad range of use cases, including deploying infrastructure for large-scale LLM training.

1. Provisioning A3-high and A3-mega clusters
Install Cluster Toolkit using the configuration instructions in the public documentation. Be sure to note some of the prerequisites including supported versions of Go, Terraform, and Packer.

Once you have a working Cluster Toolkit installation including the downloaded github repository, navigate to the examples/machine-learning blueprints directory. Here, you will have two folders for deploying H100 clusters based on the A3-series machine shapes, a3-highgpu-8g and a3-megagpu-8g. In this example, we’ll explore the blueprint in the a3-megagpu-8g folder.

Google Cloud Cluster Toolkit blueprints are Infrastructure as Code (IaC) documents that describe the infrastructure you would like to deploy, and are conceptually similar to Terraform or other IaC tooling. For the a3-megagpu-8g blueprint, there are three main files that control the deployment: 

1. slurm-a3mega-base.yaml – includes creating the necessary VPC networks along with the filestore instance used for a common home filesystem on the cluster nodes.
2. slurm-a3mega-image.yaml – creates the Compute Engine image instance that is used by Slurm to provision nodes based on the cluster’s definitio
3. slurm-a3mega-cluster.yaml – sets up the main cluster components, including the Slurm controller (the main orchestrator for Slurm jobs), the Slurm login node (a host used for job submission) and the a3mega partition (the working nodes in the cluster)

While you can customize each of the blueprint components if needed, you can easily get started by simply specifying the details for your working environment in the deployment-base.yaml and the deployment-image-cluster.yaml.

2. Enable GPUDirect-TCPXO optimized NCCL communication
Once the Slurm cluster is created, follow this tutorial to enable GPUDirect-TCPXO for optimized NCCL communication on the GPU networks. To validate the environment and ensure the TCPXO plugin is being properly loaded, build and compile the NCCL tests. Then, run sbatch run-nccl-tests.sh from the login node, being sure to change the number of nodes in the script to match those in your cluster. This runs a distributed all_gather test across the GPUs and nodes indicated in the script.

When working as intended, the NCCL tests should show output results indicating high-speed bandwidth throughput at various message sizes. A common measure of performance is to use the busbw value in GB/s from the second or last row of the output table, which shows the 4Gb and 8Gb message size values. A cluster with TCPXO active should report around 190 GB/s busbw throughput. See the performance page in the NVIDIA NCCL-tests repository for more details around these metrics.

3. Run an NeMo training workload
Follow this NeMo training tutorial to run an example NeMo Framework Pre-Training job using the following steps:

Step 1:

• Creates a NeMo Framework-derived container with the necessary TCPXO environment variables

• Submits a Slurm job to copy the framework launcher scripts and a few other auxiliary files into your working directory

Step 2:

• Establishes a Python virtual environment and installs NeMo Framework python package dependencies

Step 3:

• This command runs distributed training of a 5B parameter GPT3 model across eight nodes for 10 steps using mock data as the input.

Option 3: Vertex AI

For teams seeking managed infrastructure experience as well as access to leading open models such as Llama 3.1, Mistral, etc., Vertex AI Model Garden and Custom Training Job service presents an attractive option. This fully managed solution removes most of the orchestration burden and provides end-to-end ML platform operations, allowing you to focus on model development and experimentation. Vertex AI’s end-to-end training support further simplifies the process, offering an integrated workflow from data preparation to deployment.

Let’s look at how to perform  single-node or multi-node fine-tuning/training workload on Vertex.   

Single-node multi-GPU fine-tuning/training on Vertex
This notebook demonstrates fine-tuning and deploying Llama 3.1 models with the Vertex AI SDK. All of the examples in this notebook use parameter-efficient finetuning (PEFT) methods with Low-Rank Adaptation (LoRA) to reduce training and storage costs. LoRA is one approach of PEFT, where pretrained model weights are frozen and rank decomposition matrices representing the change in model weights are trained during fine-tuning.

Multi-node distributed fine-tuning/training on Vertex AI

This Vertex sample training repo provides examples on how to launch multi-node distributed training on A3 Mega (8 x NVIDIA H100) on Vertex. 

The NeMo example illustrates how to perform pre-training, continued pre-training and supervised fine-tuning (SFT). In addition, NeMo allows optimized training as a popular approach to evaluate the AI accelerator (A3 Mega in this case). To benchmark, you can rely on the reported metrics such as epoch time, step-time, etc. Since NeMo runs on most NVIDIA GPU types, it can be helpful for comparing different AI chips for a given task. Read on to learn how to run the example on Vertex with A3 Mega node types.

launch.sh is the main entry point to launch NeMo distributed training with command parameters:

Example:

At the end of launch.sh script, we use curl command to call Vertex customJobs API to launch NeMo training job in Vertex:

Job configurations in vertex-payload.json are part of curl command to launch Nemo training, it includes job specifications on resource requirements as showed:

The job configuration arguments “${TRANSFER_MODEL_CMD} ${LAUNCH_CMD}” in turn embed full content from the job training script, which also includes all the NCCL environments required by A3 Mega, while other pytorch launch commands are executed by Vertex CustomJob.

Optionally, build your own custom job container image as an “imageUri” parameter in vertex-payload.json, using this Dockerfile as your reference.

DIY enthusiasts: Building custom training environments

Lastly, we recognize many organizations prefer a more hands-on approach and have specific orchestration tools or frameworks that they wish to use. If that describes you, Google Compute Engine provides the foundation to build your own tailored training environments, letting you create and configure virtual machines (VMs) with your desired specifications, including the type and number of GPUs, CPU, memory, and storage. This granular control lets you optimize your infrastructure for your specific training workloads and integrate your preferred orchestration tools.

To facilitate this process, we provide example code snippets demonstrating how to use the gcloud compute instance create and gcloud compute instance bulk create API calls to create and manage your vanilla A3 Mega instances. Whether you need to create a single VM or provision a large-scale cluster, these resources can help streamline your infrastructure setup.

Conclusion

With the right orchestration strategy and Google Cloud’s robust and leading AI infrastructure, you can achieve your training goals and transform your business objectives into reality.

To learn more about distributed training, please review GKE example, Cluster Toolkit example, and Vertex AI example.

Search is at the heart of how we interact with the digital ecosystem, from online shopping to finding critical information. Enter generative AI, and user expectations are higher than ever. For applications to meet diverse user needs, they need to deliver fast, accurate and contextually relevant results, regardless of how queries are framed. For example:

• Online shoppers expect to find “waterproof hiking boots with ankle support” just as easily as a specific “SummitEdge Pro” model.

• Legal professionals need to pinpoint precise case citations or explore nuanced legal concepts with varied search terms.

• Doctors require precision when searching for critical patient information. A doctor looking for “allergy to penicillin” must locate the record accurately, whether the information is labeled as “drug sensitivities” or misspelled as “peniciln”. 

Spanner, Google’s always-on multi-model database with virtually unlimited scale, addresses these challenges with AI-powered hybrid search capabilities. Spanner allows developers to combine vector search, full-text search, and machine learning (ML) model reranking capabilities in a unified platform directly integrated with the operational data store, using a familiar SQL interface. 

In this post, we will explore how you can build a customized search engine for an ecommerce marketplace using Spanner.

Building a tailored search engine on Spanner

For ecommerce – along with many other industries – a single search method often falls short, resulting in dissatisfied users, incomplete information, or lost revenue. Keyword search excels at precision but struggles with alternate phrasing or natural language; vector search captures semantics but may overlook specific terms. Combining the strengths of both would enable organizations to deliver a more effective search experience.

SpanMart, a hypothetical ecommerce marketplace, allows users to search for products using keywords or natural language. Its products table supports multiple search methods with two specialized columns and associated indexes:

• A description_tokens column: This is a tokenized version of the description column, breaking down the text into individual terms. A search index (products_by_description) on this column accelerates full-text search, acting like an inverted index in information retrieval.

• An embedding column: This stores vector representations of the product descriptions, capturing semantic meaning rather than individual words. Similar descriptions are mapped close together in the “embedding space”. These embeddings are generated using models like Vertex AI Embeddings. A vector index (products_by_embedding) organizes these embeddings using a ScaNN tree structure for efficient semantic searches.

Here’s how the products table and its indexes are defined Spanner:

With these components in place, SpanMart can build an intelligent search pipeline that integrates: 

• Vector search for semantic relevance.
• Full-text search for precise keyword matching.
• Result fusion for combining the results from different retrieval methods. 
• ML model reranking for advanced result refinement.

This pipeline operates entirely within Spanner, where the operational data is stored. By avoiding integration with separate search engines or vector databases, Spanner eliminates the need for multiple technical stacks, complex ETL pipelines, and intricate application logic for inter-system communication. This reduces architectural and operational overhead and avoids potential performance inefficiencies.

The diagram below illustrates a high-level overview of how these components work together in Spanner.

Combining the power of vector and full-text search

When a user searches for products on SpanMart, the system first uses the embedding model to convert the user query into a vector that captures its semantic meaning. Then, SpanMart can build two queries:

• Approximate nearest neighbors (ANN) vector search:

• Full-text search (FTS):

These two queries excel in different scenarios and complement each other. For instance, when a user searches for a specific product model number, such as “Supercar T-6468”, the full-text search query can accurately find the exact model, while the vector search query suggests similar items. Conversely, for more complex natural language queries, such as “gift for an 8-year-old who enjoys logical reasoning but not a toy”, full-text search may struggle to yield useful results, whereas vector search can provide a relevant list of recommendations. Combining both queries would produce robust results for both styles of searches.

Reciprocal rank fusion (RRF)

RRF is a simple yet effective technique for combining results from multiple search queries. It calculates a relevance score for each record based on its position in all result sets, rewarding records ranked highly in individual searches. This method is particularly useful when the relevance scores from the individual searches are calculated in different spaces, making them difficult to compare directly. RRF addresses this by focusing on the relative rankings instead of scores within each result set.

Here’s how RRF works in our example:

• Calculate rank reciprocals: For each product, calculate its rank reciprocal in each result set by taking the inverse of its rank after adding a constant (e.g., 60). This constant prevents top-ranked products from dominating the final score and allows lower-ranked products to contribute meaningfully. For instance, a product ranked 5th in one result set would have a rank reciprocal of 1/(5 + 60) = 1/65 in that result set.

• Sum rank reciprocals: Sum the rank reciprocals from all result sets to get the final RRF score of a product.

The formula for RRF is –

 – where:

• d is a product description

• R is the set of retrievers (in this case, the two search queries)

• rank_r (?) is the rank of product description d in the results of retriever r.

• k is a constant

Implementing RRF within Spanner’s SQL interface is relatively straightforward. Here’s how:

Explanations:

• Common table expressions (CTEs): These are the WITH clauses, which are used in this query to improve readability. However, due to a current limitation, they may cause the query optimizer to default to an older version that lacks full-text search support. For now, the query uses the @{optimizer_version=7} hint to suggest a more recent optimizer version. 

• ANN CTE: This is the same as the previous ANN query, but with a twist. We assign a rank to each product in the results. While Spanner doesn’t support a direct way to assign ranks, there’s a workaround. By converting the results into an array of structs, we can use the offset of each element within the array as its rank. Since array offsets start at zero, we use offset + 1 to represent the actual rank. Note that this is purely a SQL language workaround without performance impact. The query planner effectively optimizes away the array conversion and directly assigns an offset to each row in the result set.

• FTS CTE: Similarly, this part mirrors the previous full-text search query, with the rank assigned using the array offset.

• Combining and ranking: The results from both CTEs are unioned, and grouped by the product id. For each product, we calculate the rrf_score and then select the top 50 products.

While RRF is an effective technique, Spanner’s versatile SQL interface empowers application developers to explore and implement various other result fusion methods. For instance, developers can normalize scores across different searches to a common range and then combine them using a weighted sum, assigning different importance to each search method. This flexibility allows for fine-grained control over the search experience and enables developers to tailor it to specific application requirements.

Using an ML model to rerank search results 

ML model-based reranking is a powerful way of refining search results to deliver improved results to the users. It applies an advanced yet computationally expensive model to a narrowed set of initial candidates, retrieved using methods like vector search, full-text search, or their combination, as discussed earlier. Due to its high computational cost, ML model-based reranking is applied after the initial retrieval reduces the result set to a small set of promising candidates.

Spanner’s integration with Vertex AI makes it possible to perform ML model-based reranking directly within Spanner. You can use a model deployed to your Vertex AI endpoint, including those available from the Vertex AI Model Garden. Once the model is deployed, you can create a corresponding reranker MODEL in Spanner.

In this example, SpanMart employs a Cross-Encoder model for reranking. This model takes two text inputs –  text and text_pair – and outputs a relevance score indicating how well the two texts align. Unlike vector search, which uses an embedding model to independently map each text into a fixed-dimensional space before measuring their similarity, a Cross-Encoder directly evaluates the two texts together. This allows the Cross-Encoder to capture richer contextual and semantic nuances in complex queries, such as “gift for an 8-year-old who enjoys logical reasoning but not a toy”. In a more advanced setup, the reranker could leverage a custom-trained model that incorporates additional signals such as product reviews, promotions, and user-specific data like browsing and purchase history, to offer an even more comprehensive search experience.

Once this model is defined in Spanner, we can proceed to add reranking on the initial search results using the following query:

Explanations:

• ANN, FTS and RRF CTEs: These are the same previously defined approximate nearest neighbors, full-text search and reciprocal rank fusion queries, respectively.

• ML.PREDICT Ranking: This step applies the reranker model to each product description as text from the RRF results, along with the search query as text_pair. The model assigns a relevance score to each product. The products are then sorted by these scores, and the top 10 are selected.

Get started

In this post, we demonstrated one approach to combine full-text search and vector search in Spanner, but developers are encouraged to explore other approaches, such as refining full-text search results with vector search, or combining multiple search results with customized fusion methods.

Learn more about Spanner and try it out today. For additional information, check out:

• Spanner vector search documentation and codelab

• Spanner full-text search overview

• Spanner vertex AI integration overview and tutorial on generating ML predictions using SQL

For enterprises, brilliance isn’t just about individual genius – it’s about the collective intelligence within an organization. But this brilliance is often hidden in silos, inaccessible to those who need it most, when they need it most. Our studies show that enterprise workers use an average of four to six tools just to ask and answer a question¹.

Generative AI holds immense promise for employee productivity. That’s why today, we’re introducing Google Agentspace. It unlocks enterprise expertise for employees with agents that bring together Gemini’s advanced reasoning, Google-quality search, and enterprise data, regardless of where it’s hosted. Google Agentspace makes your employees highly productive by helping them accomplish complex tasks that require planning, research, content generation, and actions – all with a single prompt.   

Three ways in which Google Agentspace unlocks enterprise expertise:

1. New ways to interact and engage with your enterprise data using NotebookLM: We built NotebookLM to help users make sense of complex information, and now we’re bringing this capability to enterprises. With NotebookLM Plus, your employees can upload information to synthesize, uncover insights, and enjoy new ways of engaging with data, such as podcast-like audio summaries and more. It’s the same experience millions of users love, now enhanced with security and privacy features for work. We’re also starting to roll out the experimental version of Gemini 2.0 Flash in NotebookLM. You can sign up for the early access program for NotebookLM for enterprise today. 

2. Information discovery across the enterprise: Google Agentspace gives employees a single, company-branded multimodal search agent that acts as a central source of enterprise truth for your entire organization. Building on the best of Google’s search capabilities, Agentspace can provide conversational assistance, answer complex questions, make proactive suggestions, and take actions based on your company’s unique information. Google Agentspace can do this across unstructured data – such as documents and emails – and structured data such as tables. We’ve also built translation in, so that you can understand all information, even if it originates in another language. With pre-built connectors for the most commonly used third-party applications, such as Confluence, Google Drive, Jira, Microsoft SharePoint, ServiceNow, and more, your employees can now easily access and query relevant data sources, and make better decisions.

3. Expert agents to automate your business functions: Google Agentspace is the launch point for custom AI agents that apply generative AI contextually. Now, enterprises can empower their employees in marketing, finance, legal, engineering, and more to conduct better research, draft content quickly, and automate repetitive tasks, including multi-step workflows. With Google Agentspace, enterprises can scale AI by providing a single location where employees can easily discover and access agents for their organization. Soon, employees will be able to use a low-code visual tool to build and tune their own expert agents with Google Agentspace.

With Google Agentspace, business analysts can effortlessly uncover industry trends and create compelling, data-driven presentations fueled by AI-generated insights. HR teams can revolutionize the employee experience with streamlined onboarding, even for complex tasks like 401k selection. Software engineers can proactively identify and resolve bugs, enabling them to build and iterate with greater efficiency, and accelerate deployment cycles. Marketers can unlock deeper performance analysis, optimize content recommendations, and fine-tune campaigns to achieve better results. And with multi-language support, all employees – no matter their region – can unlock enterprise expertise with agents. Sign up for the early access program. 

Finally, security is always top of mind. Google Agentspace is built on Google Cloud’s secure by design infrastructure, giving you the peace of mind to confidently deploy AI agents across your organization. Plus, we provide granular IT controls, including role-based access control (RBAC), VPC service controls, and IAM integration, ensuring your data remains protected and compliant at all times. 

Hear from our customers 

Our enterprise customers are already seeing great promise around connecting their employees with the wealth of their enterprise’s information. 

“At Deloitte, knowledge fuels our work. Our knowledge management teams are impressed with Google Agentspace’s ability to unify information scattered across various data silos. With Agentspace, our professionals can find the information they need instantly, boosting productivity and efficiency. Beyond search, the AI agentic capabilities within Google Agentspace can empower teams to take action on this knowledge, enabling us to deliver solutions and achieve client outcomes faster. In the long term, we see Agentspace as a key driver in transforming how we serve clients, fostering deeper collaboration, and unlocking new levels of insight and innovation across our organization.” – Kevin Laughridge, Alphabet Google US Lead Alliance Partner, Deloitte Consulting LLP 

“At Nokia, we create technology that helps the world act together. to connect and improve the world. Google Agentspace has the potential to revolutionize how our teams across Nokia find and leverage critical insights. We’re particularly excited by Google Agentspace’s ability to blend various data sources quotes and deliver personalized, contextually relevant answers. By unifying our knowledge resources, providing AI powered assistance and automating workflows, we strive towards reduced time spent searching for information, faster decision-making, and improved collaboration and productivity.” – Alan Triggs, Chief Digital Officer, Nokia 

“At Decathlon, we’re driven by a passion for sports and a commitment to innovation. Google Agentspace’s ability to connect teams with the right information from across organizations and provide self-service assistance is incredibly promising. Google Agentspace could become an essential enabler for product designers, marketers, and researchers, enabling them to make faster, more informed decisions and ultimately delivering even better experiences for customers. Our cross-functional teams — from data to workplace tools — see it as a great promise to meet our needs,” – Youssef BAKKALI, Group Product Manager, Decathlon

“At Onix we help businesses harness the power of technology to drive innovation and growth. Google Agentspace has immense potential to transform how enterprises deploy agents and access enterprise data at scale. By connecting disparate data sources and leveraging Google’s advanced AI for agentic workflows, Google Agentspace empowers organizations to unlock new levels of creativity. These last few months Onix has been partnering with Google to onboard early customers, and we are excited to see our clients take their first steps on this transformative journey.” – Ramnish Singh, SVP Solutions Engineering, Onix 

“Banco BV is committed to providing our team with the most innovative tools and technologies for collaboration. Google Agentspace is helping us realize the vision of enabling our employees to leverage some of the most advanced generative AI technologies for search, assistance and actions across all our critical systems in a secure and compliant way. We are particularly excited to see how our early users are leveraging Google Agentspace for faster and more comprehensive analysis and engaging with content in new multimodal ways that reflect our vision of work as more relaxed and collaborative.” – Fabio Jabur, Head of Data and AI, Banco BV 

“Google Agentspace has the potential to redefine how enterprises leverage knowledge, particularly through the power of AI agents. We’re excited to partner with Google Cloud to bring this to our customers, enabling them to securely use generative AI to enhance productivity, streamline workflows, and drive innovation. Google Agentspace’s unique ability to personalize search, provide AI assistance, and leverage innovations like NotebookLM will revolutionize how knowledge workers discover, generate, and utilize information. We believe that agents represent the future of knowledge work, and Google Agentspace is at the forefront of this exciting evolution.” – Gaurav Johar, Head of Applied AI, Quantiphi

“Our team of over 12,000 dedicated staff is united by a shared purpose: to make every day a little better for our customers by keeping daily essentials within reach for all. As Singapore’s largest food retailer, we understand that our people are the key to delivering on our social mission. To empower them with the tools they need to put their best foot forward every day, we are building an organization-wide research and assistance platform with Google Agentspace, with the aim of reducing manual effort. Google Agentspace will equip our employees with both a seamless research assistant that searches across all our documents, internal systems, and even third-party applications, and a task assistant using natural language. Our ultimate goal is to improve knowledge discovery and streamline processes across our enterprise. By enabling our people to access and share knowledge effortlessly, we can foster improved customer experiences and enhanced operational efficiency.” – Dennis Seah, Chief Digital & Technology Officer, FairPrice 

Hear from users at Deloitte Consulting LLP

“My work involves various market analyses as a strategy professional. It normally takes us a couple of weeks to read through all the research material. NotebookLM allowed us to get the initial insights in minutes and spend the time going deeper. It helps my team collaborate from a single source, pooling notes and links, freeing up more of our research time for strategic thinking and brainstorming allowing us to deliver compelling insights to our clients.” – Parinda Gandevikar, Senior Consultant, Deloitte Consulting LLP  

“NotebookLM’s podcast uncovered hidden insights in our GenAI report, revealing differences between industry sectors. Despite having read it multiple times, I had not caught this nuance. It was a real ‘aha’ moment.” – Laura Shact, Human Capital AI Lead, Deloitte Consulting LLP

“It’s a game changer for drawing insights from meetings. We took the recordings and uploaded it to NotebookLM and it did an amazing job of synthesizing the key findings.” – Tom Cahill, Senior Consultant, Deloitte Consulting LLP

“Before a long drive, I upload earnings call transcripts of recent tech companies and listen to the audio overview on my drive. It’s great how the key insights are distilled and interwoven with broader context that’s not even in those documents.” – Vivek Kulkarni, GenAI Transformation Lead, Deloitte USA LLP 

“I run a practice which involves creating and sharing a lot of different types of enablement content for the practice. It was delightful for our team to go from paging through dozens of documents to quickly finding the information we need, with the source document just one click away in NotebookLM.” – Gopal Srinivasan, Alphabet Google AI Alliance Lead, Deloitte Consulting LLP

Get started

Unlocking your organization’s collective brilliance is an investment in its future success. By connecting employees to enterprise expertise, you can create a more informed, innovative, and agile organization. 

Learn more about Google Agentspace and sign up for early access today.

^{1. Source: Survey of Knowledge Worker Communication Needs, Google Inc. (2024)}

Written by: Muhammad Umair

Here at Mandiant FLARE, malware reverse engineering is a regular part of our day jobs. At times we are required to perform basic triages on binaries, where every hour saved is critical to incident response timelines. At other times we examine complicated samples for days developing comprehensive analysis reports. As we face larger and more complex malware, often written in modern languages like Rust, knowing where to go, what to look at, and developing a “map” of the malware forms a significant effort that directly impacts our response times and triage effectiveness.

Today we introduce a new tool, XRefer (pronounced eks-reffer), which aims to shoulder some of this burden for anyone who endeavors to go down these rabbit holes like us, helping analysts get to the important parts faster while maintaining the context of their investigation.

Introduction

XRefer provides a persistent companion view to assist analysts in navigating and understanding binaries. It’s a modular and extensible tool that comes in the form of an IDA Pro plugin. Figure 1 shows the XRefer interface.

At its core, XRefer offers two complementary navigation paradigms:

1. Gemini-powered cluster analysis, which decomposes the binary into functional units and leverages the large language model (LLM) to describe their purpose and relationships. Think of this like viewing a city from Google Maps: you can quickly identify the business districts, residential areas, and green spaces. In binary terms, this feature helps identify functional groupings like command-and-control communication, persistence mechanisms, or information-gathering routines, giving you a strategic view of the malware’s architecture.

2. A context-aware view, which dynamically updates based on your current location in the code. This view presents both immediate artifacts of the current function and those from related functions along the same execution path. It’s similar to standing outside a shopping mall with X-ray vision: without entering each store, you can see the restaurant menus, shop inventories, and services offered on each floor. This allows you to make informed decisions about which areas deserve deeper investigation. Just as a mall directory helps you efficiently plan your shopping route, XRefer’s context-aware view helps analysts quickly identify relevant code paths by surfacing APIs, strings, capa matches, library information, and other artifacts that might otherwise require manual exploration of multiple functions. 

Let’s take a closer look at each of these paradigms, beginning with cluster-based navigation.

The Birds-Eye View: Cluster-Based Binary Navigation

One of XRefer’s key features is its ability to break down a binary into functional units, providing an immediate high-level understanding of its architecture. To demonstrate this capability, let’s examine an ALPHV ransomware sample written in Rust. Despite containing over 2,700 functions, XRefer’s analysis organizes key functionality of this complex binary into clear functional clusters, as shown in the Cluster Relationship graph in Figure 2.

These functional clusters are descriptively labelled as follows:

• Ransomware Main Module

• Configuration Parsing Module    

• User Profile and Process Information Module

• Privilege Escalation, System Information, and AntiAnalysis Module

• File Processing Pipeline Module

• Network Communication and Cluster Management Module

• Thread Synchronization and Keyed Events Module

• File Path and Encryption Key Generation Module

• Console Clearing Module

• UI Rendering and Console Output Module

• Image Generation and Encoding Module

• Data Encoding and Hashing Module

• File Discovery and Dispatch Module

• Thread Synchronization and Time Management Module

While each cluster contains deeper sub-clusters that analysts can explore, we’ll focus on the high-level view for now. The clustering and relationship identification is performed through static analysis. XRefer then leverages Gemini to provide natural language descriptions of each cluster and how they relate to one another. Figure 3 illustrates key components in the graph navigation interface.

At the top, the view provides a brief description of the binary’s functionality and its category. Next, it describes the currently selected cluster and its relationships to other clusters. For convenient navigation, the cross-references of that cluster are listed, followed by a visual graph representation. For readability, these details are transcribed in Table 1.

The clusters can also be viewed in a linear format within XRefer’s interface, as shown in Figure 1.

To better demonstrate cluster navigation visually, we’ve used a lightweight backdoor that displays more clearly on screen. Figure 4 provides a quick glimpse of the cluster navigation workflow, showing how analysts can quickly browse clusters and navigate to their respective functions in the disassembly or pseudocode views.

The navigation can automatically sync with clusters—when you navigate to a function that belongs to a known cluster, XRefer can automatically open that cluster’s view and highlight the current function within it. XRefer offers two approaches to clustering:

1. Cluster all paths that are part of XRefer’s analysis (XRefer’s analysis is discussed later)

2. Cluster a focused subset of functions, pre-filtered by Gemini based on their artifacts

Note: Throughout this blog post, we use the term “artifacts” to refer to binary elements like strings, API calls, library references, and other extractable information that help understand program behavior.

By default, XRefer employs the first method. While this approach is comprehensive, it may create additional clusters around unidentified libraries in the program. These library clusters are typically easy to identify and exclude from analysis.

The second clustering method is optionally available via the context menu and proves valuable for automatically filtering out library, runtime/compiler artifacts, and repetitive noisy functions. However, due to the inherent nature of LLMs, this approach can be inconsistent—artifacts might be missed, and results can vary between runs. While missed artifacts can usually be recovered through a quick re-run of the LLM analysis, this variability remains an inherent characteristic of this approach.

XRefer can also display these LLM-filtered artifacts in a dedicated view, separate from the clustering visualization. This view, shown in Figure 5, provides analysts with a streamlined overview of the binary’s most relevant artifacts while filtering out noise like library functions and runtime artifacts.

It’s important to note that clusters aren’t perfect boundaries. They may not capture every related function and can contain functions that are reused across different parts of the binary. However, any missed related functions will typically be found in the vicinity of their logical cluster, and reused functions are generally easy to identify at a glance. The goal of clustering is not to create strict divisions, but rather to establish general zones and subzones of related functionality.

Function Labeling: Prefixing Cluster Membership in Names

XRefer can optionally encode cluster information directly into IDA’s interface by prefixing function names. These labels provide architectural context directly inside the disassembly and pseudocode windows. Table 2 shows the classification system used to prefix functions based on their cluster relationships.

Down in the Trenches: Context-Aware Code Navigation

Having seen how XRefer’s cluster analysis provides a high-level view of binary architecture, let’s examine its second navigation paradigm: a context-aware view that updates automatically based on the function currently being analyzed.

The function context table (shown in Figure 6) organizes information into three main components:

1. Cluster Membership – At the top, displaying which clusters the current function belongs to. Functions appearing in multiple clusters often indicate utility or helper functions rather than specialized functionality.

2. Direct References – Listed under “DIRECT XREFS,” showing artifacts directly used or called by the current function.

3. Indirect References – Categorized tables prefixed with “INDIRECT,” showing artifacts used by all functions called through the current function’s execution paths. This provides a preview of downstream functionality without requiring manual traversal of each function.

Both direct and indirect references include:

• APIs and API traces

• Strings

• Libraries

• capa results

For direct references, each artifact is listed with its reference addresses. Double clicking these addresses jumps to their exact location in the current function. For indirect artifacts, the displayed addresses are different—they point to function calls within the current function that eventually lead to those artifacts through execution paths. This x-ray-like capability eliminates the tedious process of diving into nested function calls to discover artifacts and functionality, only to return to the primary function.

XRefer extends this visibility through its Peek View feature, accessible via the context menu. When enabled, clicking on any function dynamically filters the artifact view to display only those elements that lie along its execution paths. This instant preview allows analysts to quickly assess a function’s downstream behavior without manually tracing through its call graph, significantly streamlining the exploration of complex codebases. Figure 7 demonstrates how this functions in practice.

Beyond Peek View, XRefer offers on-demand artifact filtering through key bindings. A core design principle of XRefer is the uniform treatment of all artifact types—any operation available for one type of artifact is consistently available across all others. For instance, path analysis capabilities that work with API references can be similarly applied to capa results. Let’s examine the key functionalities available under this navigation paradigm.

Path Graphs

XRefer can generate and visualize all simple paths from the entry point to any type of artifact. These interactive graphs serve as powerful navigation tools, particularly when analysts need to trace specific functionality through the binary. Figure 8 demonstrates this capability by displaying all execution paths leading to GetComputerNameW in the ALPHV ransomware sample. Each function node in the graph provides a contextual pop-up showing its complete artifact inventory.

Path graphs include a simplification feature that can reduce visual complexity by omitting nodes that either contain no artifacts or contain only excluded artifacts (exclusions are discussed later). Figure 10 illustrates this simplification, where the graph is reduced from 15 to 12 nodes, representing a 20% reduction in complexity. While more complex graphs can achieve higher simplification ratios, their full visualization extends beyond the practical constraints of this blog post.

Search

XRefer provides unified search functionality across all artifact types directly within the function context table view. Figure 8 demonstrates this capability while searching for the API reference used in path graph generation.

Cross References++

XRefer implements its own cross-reference view that goes beyond IDA Pro’s traditional functionality (accessed via “X”). This view, similarly triggered by pressing “X” on any artifact, encompasses all artifact types, including elements that IDA Pro cannot typically track, such as capa results, APIs identified through dynamic traces, and strings extracted by language-specific modules like the Rust parser.

Trace Navigation

While API traces are integrated throughout XRefer’s interface—from function context tables (Figure 6) to information pop-ups (Figure 8)—the plugin also offers dedicated trace navigation modes. These three modes, illustrated in Figure 11, provide views of API calls with their arguments, each offering different levels of scope:

• Function Scope – Shows only the API calls made directly within the current function, providing a clean view of its immediate external interactions

• Path Scope – Reveals all API calls that occur downstream from the current function, following its execution paths. This helps analysts understand the complete chain of system interactions triggered by a particular function.

• Full Trace – Displays the complete API trace captured during dynamic analysis, regardless of static code paths. Useful when you may have calls associated with encrypted regions in the binary or generally from dynamically resolved APIs.

Artifact Exclusion

XRefer supports artifact exclusion to reduce noise when analyzing large, complex binaries. Excluded artifacts are omitted from multiple views and processes including the main interface, cluster analysis, and simplified path graphs.

Artifacts can be excluded in two ways:

• Directly through XRefer’s interface, where multiple artifacts can be selected and excluded using key bindings

• Via the settings dialog (shown in Figure 12), which supports wildcard-based exclusion patterns. For instance, noisy Rust standard library references can be filtered using patterns like std*, providing efficient bulk exclusion of known noise sources.

These exclusions persist across sessions, allowing analysts to maintain their preferred filtering setup.

Specialized Rust Support

XRefer has a specialized Rust module (discussed later) that extracts strings and library usage information. During Rust compilation, the compiler embeds library source paths that typically appear adjacent to their corresponding library code within functions. These compiler-inserted references serve two key purposes as function artifacts:

• Identifying library dependencies and their specific functionality within code sections

• Providing positional hints that help locate where library code is actually implemented within functions

This compiler-provided context feeds into both XRefer’s cluster analysis and enhances manual navigation, helping analysts quickly locate relevant code regions while understanding which Rust library implementations are being used. The module also includes a basic function renaming capability for Rust binaries, which will be covered in detail later.

Auxiliary Features

Before concluding this section, two standalone features warrant mention, though they operate independently of the clustering mechanism and exclusion system.

Boundary Method Scanning: XRefer allows multiple artifacts to be selected in the function context view for boundary scanning. This operation identifies the Lowest Common Ancestor (LCA) in the call graph for all selected artifacts. In niche scenarios, this can be used to isolate specific functionality based on a subset of artifacts and identify the most specific parent function that encompasses all selected artifacts without intermediate functions. While originally intended to serve a larger purpose in the clustering mechanism, the clustering system ultimately took a different direction, leaving this as a standalone feature.

String Lookups: During processing, XRefer can optionally query strings against public Git repositories through Grep App for categorization purposes. This is strictly a placeholder implementation; locally maintained databases by teams or individuals would be better suited for these queries. The feature operates independently of XRefer’s broader ecosystem, primarily serving to categorize known strings for noise reduction or occasional OSINT insights.

Under the Hood: XRefer’s Analysis Engine

Having explored both navigation paradigms, let’s examine the technical foundation that makes them possible. While a deep dive into XRefer’s internals would warrant its own blog post, understanding a high-level view of its analysis pipelines and extensibility will help set the context.

Ingestion Sources

XRefer builds its understanding of binaries through two primary data ingestion channels:

1. Internal – Whereby it extracts and processes all the imports, strings and any library data from the binary itself. This also involves language-specific modules. XRefer comes out of the box with a Rust-specific language module, and more can be added.

2. External – This includes API traces from third-party tooling and results from capa’s analysis. XRefer currently supports ingestion of API traces from VMRay’s sandbox and Cape Sandbox. Additional modules can be written, for instance, TTD traces would be a good candidate.

Note: Out of the mentioned traces, VMRay produces the best results. Cape sandbox, while good, hooks NT* APIs and is much noisier in terms of visual display as it loses 1:1 API to import mapping. Unfortunately, that’s one of the very few open-source solutions available right now.

XRefer feeds the available data along with cluster relationships in the form of call flows to the LLM, which generates semantic descriptions for each cluster, their relationships, and the overall binary. As demonstrated in Table 3, while external data sources enhance the analysis, XRefer can produce meaningful results even with just internal binary analysis.

Note: While LLM-generated labels and descriptions may vary in phrasing between runs, they tend to consistently convey similar semantic information.

Language Modules and Rust

XRefer supports language-specific analysis through dedicated modules and ships with a module for Rust binaries. This allows for specialized handling of language-specific characteristics. The Rust module provides:

• Identification of rust_main 

• Parsing of Rust thread objects and their indirect calls, improving path coverage

• Extraction of library/crate information for better code understanding in both cluster analysis and context tables

• Limited function renaming capabilities for a subset of functions where no inlining conflicts are detected, based on compiler strings and excluding runtime/std/internal libraries

Rust Function Renaming

The Rust module includes a limited function renaming capability for Rust binaries, though its approach is deliberately restrictive. While XRefer is not a function identification tool, it can leverage compiler strings embedded in Rust binaries for basic renaming. Due to the prevalence of inlining from compiler optimizations, the module only renames functions where no inlining is detected and excludes internal references (std*, core*, alloc*, etc.) which are particularly prone to inlining.

The renaming doesn’t provide the full function names but does include up to the submodule name. Again, this is not a substitute for proper function identification as only a very small handful of functions can be renamed this way. However, since this capability is not version-specific to the toolchain, platform, and/or crate, it represents a low-hanging fruit that would have been unwise to ignore. As an example, in the ALPHV binary, the Rust module was safely able to rename 362 out of ~2,700 functions.

LLM Analysis and Extensibility

XRefer ships with support for Gemini through a modular provider system. The architecture is designed for extensibility—new LLM providers can be added by implementing a simple interface that handles model-specific requirements like rate limiting and token management. It should be noted that cluster analysis, especially for large binaries, requires large context windows, an area in which Gemini models excel compared to other models.

Similarly, the prompt system is built to be extensible. New prompts can be easily added by creating prompt templates and corresponding response parsers. This allows XRefer’s analysis capabilities to grow as new use cases are identified or as LLM technology evolves.

It’s important to note that XRefer currently limits its LLM analysis to artifacts and function/cluster relationships (in the form of call flows) without submitting any actual code. For example, when analyzing a network communication module, XRefer provides the LLM with a rich set of artifacts like API calls (send, recv), strings (URLs, User-Agents), dynamic API traces, capa matches (network communication, socket operations), library/crate information, and their relationships in the call graph, rather than the underlying implementation code. This is just one simplified example. The actual analysis encompasses the full spectrum of artifacts XRefer collects. 

The effectiveness of this approach depends on the successful extraction of these artifacts, whether through specialized language modules, internal binary analysis, or ingestion of external data sources. When artifacts are available, the Gemini-powered analysis effectively breaks down binary functionality into distinct functional units, providing an explorable architectural view of the binary. Code-level analysis represents the next logical step in XRefer’s evolution.

Path Analysis: The Foundation of XRefer’s Navigation

XRefer’s architecture is fundamentally entrypoint-centric. It constructs execution paths between entry points and functions containing artifacts, forming the backbone of both its clustering algorithm and the context-aware navigation capabilities described earlier. While clustering can be implemented without path analysis, our current approach of path-based clustering consistently produces decent results. Standalone clustering may come as a feature later down the road.

While entry point selection for PE executables is straightforward and automatic, DLL analysis may require analysts to select specific exports as starting points, since the default entry point might not be the most interesting one. XRefer allows analysts to analyze multiple entry points, providing flexibility in how they approach the binary.

Path analysis introduces computational overhead, but the benefits it provides to both navigation capabilities and clustering accuracy make this trade-off worthwhile. It’s important to note that this overhead is entirely front-loaded into the initial processing phase. Once analysis is complete, the pre-processed results ensure fluid navigation and responsiveness during actual usage. Table 4 shows what analysis times look like for several large binaries.

Note: Times include LLM queries for artifact discovery and cluster analysis. Actual duration varies based on system capabilities and binary complexity—from seconds for simple binaries to longer for complex ones. This listing aims to provide a general sense of expected time scales.

A notable limitation of path analysis arises in binaries with numerous indirect calls, where complete coverage cannot be guaranteed without proper resolution of these indirect targets.

Configuration

All LLM configurations and paths for external data sources can be managed through XRefer’s settings dialog.

XRefer supports the ingestion of indirect cross-references that IDA cannot resolve statically. Examples can include dynamically resolved addresses for indirect calls, such as virtual function calls through vtables in C++ binaries and function pointer calls. These resolutions can be imported from sources like debugger scripts or binary instrumentation frameworks, enabling XRefer to construct more complete execution paths.

UI/UX and Compatibility

XRefer is implemented as an interactive TUI (Text-based User Interface) plugin. While IDA’s simplecustviewer_t wasn’t designed to be a proper TUI system, additional Qt implementations have been added to improve the interface. Users may encounter bugs, which will be addressed as they are reported.

Some rules of thumb when working with XRefer:

• All addresses, regardless of where they are displayed in the TUI, are navigable by double clicking.

• Cluster IDs (cluster.id.xxxx), regardless of where they show up in the TUI, are also explorable by single clicking.

• If an address is the start of a function, hovering over it will always display an information pop-up about that function.

• Hovering over cluster IDs will display a pop-up with details about that cluster.

• Press ESC to return to the previous location/state (unless a graph is explicitly “pinned”).

XRefer maintains state within the current function and resets upon navigation to a new function. For detailed usage instructions and key bindings, please refer to the XRefer repository.

It is recommended to enable auto-resizing, which automatically adjusts XRefer’s window dimensions when viewing graphs and restores default size upon exit. While XRefer is designed to remain open as a persistent companion view, it includes an expand/collapse feature for quick access when needed. Figure 17 demonstrates these interface elements.

XRefer is recommended for use with either IDA <= 8.3 or IDA >= 9.0. IDA 8.4 contains a simplecustviewer_t visual bug that causes washed-out colors in several areas. Due primarily to the author’s preference for dark mode, the plugin currently provides better color contrast in dark themes compared to the default theme, though this may be balanced in future releases.

XRefer has been primarily tested with Windows binaries. While there are no fundamental limitations preventing support for ELF or Mach-O formats and XRefer may just work out of the box with most of them, some tweaks or implementation fixes might be required to ensure proper support.

How to Get XRefer

XRefer is now available as an open-source tool in Mandiant’s GitHub repository. Installation requires installing Python dependencies from requirements.txt and copying the XRefer plugin to IDA’s plugin directory.

Note that one of the dependencies, asciinet, requires Java (JRE or OpenJDK) to be configured on your system. For detailed installation instructions, please refer to the XRefer repository.

Another alternative is to use FLARE-VM, which sets up a reverse engineering environment with a lot of useful tools, including XRefer.

Future Work

This is the initial release of XRefer and thus includes some implementations that are early in their maturity. While LLMs may eventually evolve to accurately interpret all forms of source and compiled code, the current approach focuses on systematic analysis rather than treating LLMs as a black box for binary summarization. This methodology not only provides high-level insights but actively supports analysts in their detailed investigation workflows.

Immediate areas for future development include, beyond bug fixes and UI/UX refinements:

• Extend cluster analysis to include code submissions, improving not just analysis at scale but also providing targeted insights for manual reverse engineering workflows

• Research and potentially implement path-independent clustering methodologies (the primary benefit here would be speed improvements if path analysis is not required)

• Implement LLM-based cluster merging (helps neatly tuck away similar clusters such as those belonging to a library)

• Ensure proper support for non-Windows file formats

• Add support for other language modules, particularly Golang

Currently, XRefer is tightly coupled with IDA due to its TUI implementation. As the core matures, the processing engine may be decoupled into a standalone package.

Acknowledgements

Special thanks to Genwei Jiang and Mustafa Nasser for their code contributions to XRefer and to Ana Martinez Gomez for including XRefer in the default FLARE-VM configuration. Additional thanks to the FLARE team members who provided valuable feedback through their use of XRefer.

The AI phase of industrial evolution is marked by a profound transformation in how humans and intelligent machines collaborate. The blurring of boundaries between physical and digital systems across the manufacturing landscape is accelerating, driven by advancements in automation, robotics, artificial intelligence, and the Internet of Things. 

This interconnectedness creates unprecedented opportunities for efficiency, innovation, and customized production. However, it also exposes manufacturers to a new generation of cyber threats targeting industrial operations, supply chains, and increasingly-sophisticated production processes. Safeguarding these critical assets requires a holistic approach that transcends traditional boundaries and embraces sector-wide collaboration. 

To enhance our commitment to the manufacturing and industry sector, today we are announcing a new partnership with the Global Resilience Federation (GRF) by joining four of its affiliate groups: the Business Resilience Council (BRC), the Manufacturing Information Sharing and Analysis Center (MFG-ISAC), the Operational Technology Information Sharing and Analysis Center (OT-ISAC), and the Energy Analytic Security Exchange (EASE). Google Cloud is proud to be the first cloud service provider to partner with the GRF Business Resilience Council and its affiliates.

Through this partnership, Google Cloud will strengthen its commitment to the manufacturing industry by providing critical expertise and advanced security solutions. Our collaboration with industry leaders will focus on fortifying the resilience of manufacturing systems and supply chains against evolving cyber threats. This partnership underscores our dedication to supporting the manufacturing sector’s digital transformation and modernization while ensuring the security and integrity of critical infrastructure. 

In today’s interconnected world, safeguarding your organization demands a comprehensive strategy that goes beyond traditional measures. Google Cloud will devote resources and experts to work alongside industry leaders to transform, secure, and defend the Manufacturing  sector and will contribute to the manufacturing companies through a network of resources and expertise spanning IT, OT, industrial operations technology, supply chain, logistics, engineering technology, and product security, specifically designed to navigate the complexities of Industry 4.0 and 5.0.

This collaboration among professionals in cyber and physical security, geopolitical risk, business continuity, disaster recovery, and third-party risk management is critical for organizations with regional, national, and international footprints. In an era where the severity of cyber threats is constantly increasing, resilience is key. Partnerships fostered by GRF provide the knowledge and support necessary to maintain vigilance, manage crises, and navigate response scenarios to enable continuity of your operations.

As a GRF partner and a member of these four groups, Google Cloud will bring experts and resources — including unique insights from Mandiant, our Threat Horizon reports, and the Google Cloud Office of the CISO— to help the manufacturing and industry sector protect against cyberattacks. Google will work with defenders and sector leaders to share knowledge we’ve learned building and deploying secure technology.

This partnership is a continuation of our August 2021 commitment to invest at least $10 billion over five years to advance cybersecurity. This same commitment has enabled us to join other organizations including Health ISAC, Financial Services ISAC, and Electricity ISAC, so we can continue to support the security and resilience of our critical infrastructure across key sectors.

“Partnering with GRF and becoming a member of its affiliated groups BRC, MFG-ISAC, OT-ISAC, and EASE is a critical step in our commitment to help the manufacturing and industrial sectors transform and secure their critical infrastructure,” said Phil Venables, VP and CISO, Google Cloud. “As a leading provider of cloud technologies and security solutions, we recognize the vital role these sectors play in driving economic growth and innovation. This partnership aligns with our dedication to supporting the modernization and resilience of manufacturing and industrial operations in the face of evolving cyber threats. By sharing our expertise and collaborating with industry leaders, we aim to raise awareness, develop innovative solutions, and strengthen the collective defense of these essential industries.”

“As a provider of innovative technology solutions, we recognize the vital role of the manufacturing and industrial sectors in driving our economy. This partnership reflects our commitment to supporting their transformation and strengthening their defenses against evolving cyber threats. Through collaboration and knowledge-sharing, we aim to foster a more secure and resilient future for these essential sectors,” said Nick Godfrey, senior director and global head, Office of the CISO, Google Cloud.

“Phil Venables and Google Cloud have long advocated for collaborative security and collective resilience, and their active role in the BRC and these communities brings invaluable expertise to help build a more secure ecosystem for businesses of all sizes — including their critical vendors and suppliers,” said Mark Orsi, CEO, GRF. “Google Cloud continues its leadership in advancing security and operational resilience across manufacturing, utilities, industrial, and critical infrastructure sectors — ultimately fostering a safer and more sustainable global supply chain.”

Your business data sets you apart from the competition. It fuels your innovations, your culture, and provides all your employees a foundation from which to build and explore. Since 2022, enterprises in all industries have turned to Looker Studio Pro to empower their businesses with self-service dashboards and AI-driven visualizations and insights, complete with advanced enterprise capabilities and Google Cloud technical support.

As the Looker community has grown, we’ve gotten more requests for guidance on how users can make their Looker Studio Pro environments even stronger, and tap into more sophisticated features. Those requests have only increased, accelerated by the debut of Studio in Looker, which brings Looker Studio Pro to the broader Looker platform. To help, today we are debuting a new on-demand training course: Looker Studio Pro Essentials.

Looker Studio Pro connects businesses’ need to govern data access with individual employees’ needs to explore, build and ask questions. This Google Cloud Skills Boost course helps users go beyond the basics of setting up reports and visualizations, and provides a deep dive into Looker Studio Pro’s more powerful features and capabilities.

Here’s what you can expect to get from this course:

• Gain a comprehensive understanding of Looker Studio Pro: Explore its key features and functionality, and discover how it elevates your data analysis capabilities.

• Enhance collaboration: Learn how to create and manage collaborative workspaces, streamline report sharing, and automate report delivery.

• Schedule and share reports: Learn how to customize scheduling options to your business, including delivery of reports to multiple recipients via Google Chat and email, based on your sharing preferences.

• Ensure data security and control: Become an expert in user management, audit log monitoring, and other essential administrative tasks that can help you maintain data integrity.

• Leverage Google Cloud customer care: Learn how to use Google Cloud Customer Care resources to find solutions, report issues, and provide feedback.

From your focus, to your employees, to your customers, your business is unique. That’s why we designed this course to bring value to everyone — from sales and marketing professionals, to data analysts, to product innovators — providing them with the knowledge and skills they need to fully leverage Looker Studio Pro in their own environments. Because in the gen AI era, how you leverage your data and invigorate your employees to do more is the true opportunity. Accelerate that opportunity with the new Looker Studio Pro Essentials course today.

For developers and businesses that run applications on Google Kubernetes Engine (GKE), scaling deployments down to zero when they are idle can offer significant financial savings. GKE’s Cluster Autoscaler efficiently manages node pool sizes, but for applications that require complete shutdown and startup (scaling the node pool all the way to and from zero), you need an alternative, as GKE doesn’t natively offer scale-to-zero functionality. This is important for applications with intermittent workloads or varying traffic patterns. 

In this blog post, we demonstrate how to integrate the open-source Kubernetes Event-driven Autoscaler (KEDA) to achieve this. With KEDA, you can align your costs directly with your needs, paying only for the resources consumed.

Why scale to zero?

Minimizing costs is a primary driver for scaling to zero, and applies to a wide variety of scenarios. For technical experts, this is particularly crucial when dealing with:

• GPU-intensive workloads: AI/ML workloads often require powerful GPUs, which can be expensive to keep running even when idle.

• Applications with predictable downtime: Internal tools with specific usage hours — scale down resources for applications used only during business hours or specific days of the week.

• Seasonal applications: Scale to zero during the off-season for applications with predictable periods of low activity.

• On-demand staging environments: Replicate production environments for testing and validation, scaling them to zero after testing is complete.

• Development, demo and proof-of-concept environments:

  • • Short-term demonstrations: Showcase applications or features to clients or stakeholders, scaling down resources after the demonstration.

    • Temporary proof-of-concept deployments: Test new ideas or technologies in a live environment, scaling to zero after evaluation.

    • Development environment: Spin up resources for testing, code reviews, or feature branches and scale them down to zero when not needed, optimizing costs for temporary workloads.

• Event-driven applications:

• Microservices with sporadic traffic: Scale individual services to zero when they are idle and automatically scale them up when requests arrive, optimizing resource utilization for unpredictable traffic patterns.

• Serverless functions: Execute code in response to events without managing servers, automatically scaling to zero when inactive.

Introducing KEDA for GKE

KEDA is an open-source, Kubernetes-native solution that enables you to scale deployments based on a variety of metrics and events. KEDA can trigger scaling actions based on external events such as message queue depth or incoming HTTP requests. And unlike the current implementation of Horizontal Pod Autoscaler (HPA), KEDA supports scaling workloads to zero, making it a strong choice for handling intermittent jobs or applications with fluctuating demand.

Use cases

Let’s explore two common scenarios where KEDA’s scale-to-zero capabilities are beneficial:

1. Scaling a Pub/Sub worker

• Scenario: A deployment processes messages from a Pub/Sub topic. When no messages are available, scaling down to zero saves resources and costs.

• Solution: KEDA’s Pub/Sub scaler monitors the message queue and triggers scaling actions accordingly. By configuring a ScaledObject resource, you can specify that the deployment scales down to zero replicas when the queue is empty.

2. Scaling a GPU-dependent workload, such as an Ollama deployment for LLM serving

• Scenario: An Ollama-based large language model (LLM) performs inference tasks. To minimize GPU usage and costs, the deployment needs to scale down to zero when there are no inference requests.

• Solution: Combining HTTP-KEDA (a beta feature of KEDA) with Ollama enables scale-to-zero functionality. HTTP-KEDA scales deployments based on HTTP request metrics, while Ollama serves the LLM.

Get started with KEDA on GKE

KEDA offers a powerful and flexible solution for achieving scale-to-zero functionality on GKE. By leveraging KEDA’s event-driven scaling capabilities, you can optimize resource utilization, minimize costs, and improve the efficiency of your Kubernetes deployments. Please remember to validate usage scenarios as scale to zero mechanism can influence workload performance. Scaling to zero can increase latency due to cold starts. When an application scales to zero, it means there are no running instances. When a request comes in, a new instance has to be started, increasing latency. 

There are also considerations about state management. When instances are terminated, any in-memory state is lost.

To help you get started quickly, we’ve published a guide for scaling GKE to zero with KEDA, with specific steps for scaling a Pub/Sub worker to zero with KEDA on GKE and scaling a Ollama to zero with KEDA on GKE. 

To learn more about KEDA and its various scalers, refer to the official KEDA documentation at https://keda.sh/docs/latest.

Dun & Bradstreet, a leading global provider of business data and analytics, is committed to maintaining its position at the forefront of innovation. For the past two years, this commitment has included the company’s deliberate approach to improving its software development lifecycle by infusing AI solutions. 

While development velocity and security were the company’s most pressing considerations, Dun & Bradstreet was also inundated with productivity and operational challenges common to many global enterprises which included: 

• Significant time onboarding new team members 

• Siloed knowledge of legacy codebases 

• Low test coverage

• Application modernization challenges

To achieve its goal of accelerating software development, Dun & Bradstreet knew it had to take a holistic “people, process, and tools” approach to solve the traditional development lifecycle issues that most enterprise engineering teams face. They looked to AI-assistance to anchor this new effort.

Finding a partner for the future of the software development lifecycle

As a provider of information that can move markets and drive economies, Dun & Bradstreet had a high bar for any technology tools, with demanding expectations as high as the financial professionals and government leaders they serve. 

Dun & Bradstreet executed a thorough evaluation process to identify the best partner and coding assistance tool, considering both open-source and commercial options. The company ultimately selected Gemini Code Assist due to the Gemini model’s performance, seamless integration with their existing development environment, and robust security features.

The implementation of Gemini Code Assist was a collaborative effort between Dun & Bradstreet’s development teams and the Google Cloud team. The developers who were part of the team were actively involved in the configuration and customization of the tool to ensure that it met their specific needs and workflows.

A key focus area for Dun & Bradstreet was Google’s security stance. Incorporating AI into the development process required both top-grade protection of private data and guardrails to ensure the safety of machine-generated code. Google’s security expertise and guidance allowed Dun & Bradstreet to move forward with confidence due to the following factors:

• Gemini models are built in-house, allowing Google to fully validate and filter all source code samples used in model training.

• Trust and verify: Integration into a company’s existing coding and review lifecycles allows developers to guide the model outputs with human oversight, without learning a whole new system.

• Google’s partnership with Snyk provides additional options for automated security scanning, covering both AI-generated and human-written code.

• Google’s AI Principles underpin the architecture and design decisions for Gemini Code Assist. Privacy and security protections include single-tenant storage of customer code references, encrypted logging, and fine-grained administrative controls to prevent accidental data leakage.

• Google’s indemnification policies.

“AI-assisted code creation is not just a leap forward in efficiency — it’s a testament to how innovation and security can work hand-in-hand to drive business success,” said Jay DePaul, chief cybersecurity technology risk officer at Dun & Bradstreet. “By embedding robust guardrails, we’re enabling our teams to build faster, safer, and smarter.”

Transformation in action

Dun & Bradstreet decided to move forward with Code Assist in October 2024. The solution is now starting to roll out to more teams across the organization. Adoption has been smooth, aided by Code Assist’s intuitive interface and comprehensive documentation.

Having a program for incubation at large organizations helps to iron out both technical and potential adoption blockers. For example, the Dun & Bradstreet team identified the need to educate teams on how coding assistants are there to help developers as a partner, not as replacements.

Now that the rollout is underway, Dun & Bradstreet is sharing the factors that drove their adoption of Gemini Code Assist.

• Increased developer productivity: Gemini Code Assist’s AI-powered code suggestions and completions have significantly reduced the time developers spend writing code. The tool’s ability to automate repetitive tasks has freed up time for the developers so they can focus on more complex and creative aspects of their work.

• Improved code quality: The automated code review and linting capabilities of Gemini Code Assist helped Dun & Bradstreet’s developers detect errors and potential issues early in the development process. This has led to a significant reduction in bugs and improved overall code quality.

• Easier application modernization: A significant amount of time was saved when converting Spring apps to Kotlin.

• Increased developer efficiency: Early internal indicators show a 30% increase in developer productivity.

• Developer onboarding: New developers at Dun & Bradstreet have been able to ramp up quicker due to the real-time guidance and support provided by Gemini Code Assist. 

• Enhanced knowledge sharing: Gemini Code Assist has fostered a culture of knowledge sharing within Dun & Bradstreet’s development teams. The tool’s ability to provide code examples and best practices made it easier for developers to learn from each other and collaborate effectively.

Leading the way with AI

Gemini Code Assist has proven to be a valuable solution for Dun & Bradstreet as it has empowered their developers with advanced AI capabilities and intelligent code assistance. 

“AI-assisted code creation is a game changer for everyone involved in the solution-delivery business,” said Adam Fayne, vice president for Enterprise Engineering at Dun & Bradstreet. “It enables our teams to innovate, test, and deploy faster, without having to risk security or quality.”

The company has been able to accelerate velocity, improve software quality, and maintain its competitive edge in the market. Companies like Dun & Bradstreet trust Google Cloud and Gemini to greatly enhance their software developer lifecycles. In fact, Google Cloud was recently named a Leader in the 2024 Gartner Magic Quadrant for AI Code Assistants for its Completeness of Vision and Ability to Execute.

Visit the Gemini Code Assist website for more information on use cases and how to start using it today.