When meeting with Communications Service Providers (CSPs) around the world, there is one clear theme: artificial intelligence (AI) is changing the game, unlocking significant value across the CSP value chain, and changing how CSPs and their customers interact with technology.

MWC’24 is no exception, and we are thrilled to be back in Barcelona this week as the wider industry ecosystem comes together to reimagine the future of the industry, with AI-driven innovation at its core.

It’s been an exciting 12 months since last year’s event — we have been collaborating with CSP customers and partners around the world to digitally transform their business with data and AI. For example:

Accenture, the leading professional services firm, and Google Cloud committed to establishing a Joint Generative AI Center of Excellence, with access to over 130 models through Model Garden on Vertex AI, and the ability to complete rapid prototyping of industry use cases.Bell Canada is leveraging conversational analytics, Contact Center AI (CCAI) to improve the customer experience. In an effort to continue to drive innovation and customer experience improvements, Bell Canada will explore generative AI across personalization, marketing, and customer service.BT Group announced a new strategic security partnership with us, targeting a range of joint innovation opportunities.KDDI will launch an initiative providing services that enrich the lives of customers by enabling them to experience gen AI in their daily lives. KDDI will combine its metaverse and Web3 service αU (Alpha-U) with gen AI, including Google’s high-performance gen AI model Gemini Pro and Immersive Stream for XR.Maxis has expanded its strategic collaboration with Google Cloud by integrating enterprise-grade gen AI into its internal workflows and customer service offerings. It is leveraging Duet AI for Developers for AI-powered coding assistance in natural language, and multimodal capabilities from the Gemini models on Vertex AI. This builds on Maxis’ existing use of Google Cloud’s unified data management and machine learning development platforms, empowering it to further streamline its operations, more quickly bring new digital services to market, and create more differentiated customer experiences.Nokia launched their AVA Data Suite to run on Google Cloud and help CSPs standardize data and facilitate the development of AI and machine learning (ML) software applications, including secure autonomous operations and predictive network maintenance.Smart Communications, Inc. (Smart), the mobile services arm of PLDT Group, has adopted Google Cloud’s Telecom Subscriber Insights, which will help inform the development of services that equip Filipinos to better understand, manage, and optimize their mobile data consumption, and also enable Smart to design more inclusive mobile services for subscribers by identifying both patterns and gaps in connectivity.TM Forum and Google Cloud, along with Accenture, Deutsche Telekom, Reliance Jio, Orange, and Vodafone came together to launch the TM Forum Innovation Hub, with a specific focus during the pilot phase on the potential of gen AI and large language models for CSPs; as well as accelerating the Open Digital Architecture (ODA).

And to support our CSP customers build, deploy, and operate hybrid cloud-native networks:

We are announcing the general availability of Google Cloud’s Telecom Network Automation to provide intent-driven automation for Telecom networks based on the Nephio open-source project. The product is now available from the Google Cloud console.Ericsson has expanded its successful collaboration with Google Cloud to develop an Ericsson Cloud RAN solution on Google Distributed Cloud that offers integrated automation and orchestration, and leverages AI/ML for additional benefits to CSPs.‘’Orange sees GDC Edge with Dataproc and Data Analytics as a perfect solution to analyze customer and network data in a secure on-premise environment across our 26 countries. Google GDC Edge offers Orange an elegant extension from public cloud to on-premise of our tools, employee skills, and operating model.’’ (Steve Jarrett, Chief AI Officer, Orange)Telefónica Germany is partnering with Google Cloud and Nokia to integrate enhanced energy efficiency into their networks.

CSPs in the gen AI era

Gen AI heralds the next pivotal moment in our industry’s AI journey — and will be front and center at MWC’24.

Advancing the innovation of the future AI-enabled telco is Gemini, Google’s largest and most capable AI model. Earlier this month we announced the next chapter of our Gemini era, introducing Gemini Advanced. We are also bringing Gemini’s capabilities to more products including Workspace and Google Cloud — helping companies to boost productivity, developers to code faster, and organizations to protect themselves from cyber attacks, along with other benefits.

And most recently we announced our next-generation model Gemini 1.5, which delivers dramatically enhanced performance with a breakthrough in long-context understanding across modalities, as well as Gemma, a family of lightweight, state-of-the art open models built from the same research and technology used to create the Gemini models.

As CSPs harness the full potential of gen AI to drive value across their organizations, gen AI assistants will play critical roles, augmenting human capabilities with powerful multi-modal data analysis, pattern recognition, and recommendations, all leveraging historical CSP knowledge and using industry best practices to streamline processes and transform customer and employee experiences. We see four key areas where gen AI agents have a critical role to play:

Automating network operations
Telecommunication providers are undergoing a profound transformation, migrating towards cloud-native architectures and intent-based networking automation. This shift lays the foundation for the next generation of networks, characterized by enhanced AI capabilities and increased autonomy. Together, these advancements promise a future where networks can self-optimize, self-heal, and proactively adapt to changing conditions, delivering new levels of reliability and efficiency for telecom providers.

In this new phase of AI, we see gen AI agents having a tremendous opportunity to transform network planning, engineering, deployment, and operations. For example, they could provide engineer assistance via an accessible knowledge base to accelerate learning, generate and optimize network intent and policies, troubleshoot and provide recommendations, and help to quickly triage faults to ensure high network uptime.

Accelerating field services
As networks have become distributed, telecom field services have evolved drastically to maintain network quality. Early services focused on reactive responses — technicians were dispatched only after a failure caused an outage. Later, preventative maintenance improved things somewhat, but relied on scheduled work and workforce expertise. Today, gen AI is helping to unlock significant value for CSPs to improve field operations, which in turn can help them lower costs and increase customer satisfaction.

Multimodal gen AI agents — with their ability to understand and reason about text, images, video, and code — are unleashing exciting possibilities for improving field operations. For example, gen AI agents can help CSPs unlock new value across workforce tasks, skillset gaps, technician assist (e.g. multimodal interactions for troubleshooting), and ticket annotation.

Improving customer care
The use of AI in contact centers has increased both customer and employee satisfaction, providing customers with fast, helpful answers from well-trained virtual agents, as well as from human agents who receive assistance from AI-powered tools. This is driving reductions in call resolution time and average handling time, ultimately resulting in lower call queues and improved net promoter score (NPS).

Gen AI has widened virtual agents’ scope, encompassing more of the customer user journey and driving not just operational efficiency, but also top-line revenue across sales (product recommendations), activation, and retention. Gen AI has also lowered the barrier to entry and time to value in the contact center in meaningful ways: more interactions can be automated in shorter periods of time, human agents can be trained and coached faster, and call insights can be observed and reacted to more effectively.

Transforming sales and marketing
Customer lifetime value starts by building customer experiences with highly personalized interactions that retain, upsell, and drive next best actions and new service offerings. This requires creating better campaigns with AI-driven segmentation, identifying customers with a high propensity for conversion, and generating targeted messages across channels. For example, integrated AI assistants can develop hyper-personalized experiences that allow the CSP to better engage with their customers, leveraging past buy propensities, customer experiences, issues, and call records.

Gen AI further enriches the highly personalized outreach with customers through their smartphone devices. From offers to campaigns, CSPs can enhance the effectiveness by generating rich content with text, images and videos. For example, Google’s Imagen can create high resolution images from user’s textual prompts and experience patterns into various styles aligned to the CSPs brand guidelines. Better contextualization and content can be integrated to drive richer advertising, including Google Ads.

Showcasing the power of gen AI at MWC

We look forward to seeing you on the ground in Barcelona! Be sure to join Google DeepMind CEO Demis Hassabis and renowned tech journalist Steven Levy on 26 February from 3:15-3:45pm on the MWC Main Stage for a fireside chat around how Artificial Intelligence can change the world, from tackling major scientific problems like energy, climate change and drug discovery, to transforming the way people create, communicate, and do business.

We’re also excited to bring a dynamic demo showcase to this year’s MWC, providing real-world examples of how Google Cloud and our gen AI portfolio are driving the next phase of cloud transformation and innovation for CSPs, including use cases across network transformation, AI-powered customer experiences, and monetization. Be sure to reach out to your dedicated Google Cloud representative to schedule a demo tour.

In addition, we’re proud to collaborate with our ecosystem of partners who will showcase CSP-focussed AI use cases, including gen AI, in their booths. This impressive lineup of Global System Integrators, ISV Partners, and Network Equipment Providers include Accenture, Anritsu, Capgemini, Carto, Datatonic, Dell, Ericsson, Intel, Nokia, Optiva, Radcom, Subex, Unacast, Viavi Solutions, and more.

For more information on how Google Cloud is partnering with CSPs on their journey to the AI-enabled telco, click here.

Securing cloud credentials has emerged as a challenge on the scale of Moby Dick: It presents an enormous problem, and simple solutions remain elusive. Credential security problems are also widespread: More than 69% of cloud compromises were caused by credential issues, including weak passwords, no passwords, and exposed APIs, according to Google Cloud’s Q3 2023 Threat Horizons Report. However, unlike Moby Dick, this story may have a happy ending. Organizations can get started on reducing the risk they face from credential-related compromises by protecting service accounts.

Service accounts are essential tools in cloud management. They make API calls on behalf of applications and they rely on IAM roles to make those calls.They also make an appealing target for attackers to establish initial access in a cloud environment.

“Nearly 65% of alerts across organizations were related to risky use of service accounts. These accounts have associated permissions where, if compromised, could lead to attackers gaining persistence and subsequently using this access for privilege escalation in cloud environments,” we wrote in the report.

This assessment follows a similar conclusion from our Q1 2023 Threat Horizons Report, where we detailed how attackers can abuse service account keys — and what organizations can do to stop them. That same report noted that 68% of service accounts had overly permissive roles, service account keys are often found hardcoded on public repositories, and project owners did not take corrective action after Google attempted to contact them in 42% of leaked key incidents.

Here are some mitigation tips that can help you avoid creating service account keys at scale, monitor for key usage, and respond to alerts quickly. We strongly encourage you to assess all of the techniques highlighted in the Q1 2023 report.

Prevent service account key creation

This section assumes that you have the necessary permissions to manage Organization Policies, guardrails that set broad yet unbendable limits for cloud engineers before they start creating and working with resources. To learn more, see creating and managing organization policies.

One of the best practices for managing service account keys is to use Organization Policy constraints to prevent creating new service account keys, and allow exceptions only for projects that have demonstrated that they cannot use a more secure alternative.

The easiest way to enforce this Organization Policy Constraint is in your Google Cloud console.

Before proceeding, ensure that you are logged in to the correct Google account. If you want to enforce this constraint across the entire Organization, be sure to select the Organization in the Resource Manager explorer:

Then click on “Manage Policy”:

And enforce the policy:

Subsequently, any time a user attempts to create a service account key, the user will be presented with the following error:

Find instances of Org Policy Changes and Service Account Key Creation

Now that service account key creation is disabled, the next step is to ensure that no one has altered this policy and covertly created new service account keys.

You can find evidence of policy changes using the Google Cloud Operations Suite. These next two examples assume that all logs are stored in a centralized Google Cloud Project and that you have the required permissions to query logs and set alerts. To learn more, see:

Routing and Storage OverviewCollate and route organization-level logs to supported destinationsBuild queries by using the Logging query languageConfigure log-based alerts

Visit the Logs Explorer. Use the Refine scope button in the Action toolbar and select the Scope that represents the location of your Organization’s centralized logs.

Use the following query to find instances of the “iam.disableServiceAccountKeyCreation” Organization Constraint being changed:

Log entry of an Org Policy Change Affecting “iam.disableServiceAccountKeyCreation”

To create an alert based on this query, proceed to the “Create Alerts for Rapid Response” section.

Find instances of service account key creation

Visit the Logs Explorer. Use the Refine scope button in the Action toolbar and select the Scope that represents the location of your Organization’s centralized logs.

Use the following query to find instances of a new service account key:

This log entry signifies that:

At some point, the Org Policy Constraint preventing service account key creation was disabledSomeone created a new service account key

Create alerts for rapid response

Google Cloud Operations Suite

The previous sections described how you can find instances of either new service account keys or Organization Policy Changes. To promote rapid and reactive response, create alerts so you may respond quickly to such events.

This section assumes you understand the following:

How Alerting WorksCreate and Manage Notification ChannelsManage Incidents for Log-Based Alerts

Click on “Create Alert”

Name the Alert and confirm that the logs to include in the alert match the query in the Log Explorer. Click “Preview Logs” to confirm the entries are desirable. Set the notification frequency and autoclose duration.

Choose the notification channel. If you need to create a notification channel, follow these instructions.

Below is an example of an alert, sent via email, listing an incident triggered after I created a service account key.

Clicking “View Incident” directs me to the Alerting page:

Clicking on the incident directs me to a view listing:

the conditionthe log querythe set of logs matching the querythe date and time the incident was triggered

Next steps: Build a collaborative incident management process

In examining the risks of using service account keys, enforcing Organization Policies that limit service account key creation at scale, and using continuous monitoring to detect policy violations, we have shown that you can better manage one of the biggest potential risks in your environment.

We highly encourage everyone to read “Build a Collaborative Incident Management Process,” which can help your organization operationalize responses to this and other types of incidents. Besides, who needs the drama of chasing a great white whale? Ditching service account keys is a much simpler way to boost your cloud security.

Let’s break down how Private Service Connect works to access a published service based on an internal network-passthrough load balancer using a connect endpoint. In this scenario, you set up a Private Service Connect endpoint on the consumer side by configuring a forwarding rule that targets a service attachment. This endpoint has an IP address within your VPC network.

When a VM instance in the VPC network sends traffic to this endpoint, the host’s networking stack will apply client-side load balancing to send the traffic to a destination host based on the location, load and health.The packets are encapsulated and routed through Google Cloud’s network fabric.At the destination host, the packet processor will apply Source Network Address Translation (SNAT) and Destination Network Address Translation (DNAT) using the NAT subnet configured and the producer IP address of the service, respectively.The packet is delivered to the VM instance serving as the load balancer’s backend.

All of this is orchestrated by Andromeda’s control plane; with a few exceptions, there are no middle box or intermediaries involved in this process, enabling you to achieve line rate performance. For additional details, see Private Service Connect architecture and performance.

With this background, you should be already able to identify the main components where issues could occur: the source host, the network fabric, the destination host, and the control-plane.

Know your troubleshooting tools

The Google Cloud console provides you with the following tools to troubleshoot most of the Private Service Connect issues that you might encounter.

Connectivity Test

Connectivity Tests is a diagnostics tool that lets you check connectivity between network endpoints. It analyzes your configuration and, in some cases, performs live data-plane analysis between the endpoints.

Producers can enable VPC Flow Logs at the target load balancer subnet to monitor traffic ingressing their VM instances backends.

Consider that VPC Flow Logs are sampled and may not capture short-lived connections. To get more detailed information, run a packet capture using tcpdump.

For more information, see Monitor Private Service Connect connections.

TIP: Be proactive and set alerts to inform you when you are close to exhausting a known limit (including Private Service Connect quotas).

In this example, you can use this MQL query to track PSC Internal LB Forwarding Rules quota usage.

Read the manual

Consult the Google Cloud documentation to learn about the limitations and supported configurations.

Follow the Private Service Connect guides.
Especially for new deployments, it is common to misconfigure a component or find that it is not compatible or supported yet. Ensure that you have gone through the right configuration steps, and go through the limitations and compatibility matrix.Take a look at the VPC Release notes.
See if there are any known issues related to Private Service Connect, and look for any new features that could have introduced unwanted behavior.

Common issues

Selecting the right tool depends on the specific situation you encounter and where you are in the life cycle of your Private Service Connect journey. Before you start, gather consumer and producer project details, and that in fact, this is a Private Service Connect issue, and not a Private services access problem.

Generally, you can face issues during setup or update of any related component or additional capability, or the issues could be present during runtime, when everything is configured but you run into connectivity or performance issues.

Issues during setup

Make sure that you are following the configuration guide and you have an understanding of the scope and limitations.

Check for any error message or warning in the Logs Explorer.Verify that the setup is compatible and supported as per the configuration guides.See if there is any related quota exceeded like the Private Service Connect forwarding rules.Confirm whether there is an organization policy that could prevent the configuration of Private Service Connect components.

Issues during runtime

Isolate the issue to the consumer or the producer side of the connection.

If you are on the consumer side, check if your endpoint or backend is accepted in the connection status at the Private Service Connect page. Otherwise, review in the producer side the accept/reject connection list and the connection reconciliation setup.If your endpoint is unreachable, check bypassing DNS resolution and run a Connectivity Test to validate routes and firewalls from the source endpoint IP address to the PSC endpoint as destination. On the service producer side, check if the producer service is reachable within the producer VPC network, and from an IP address in the Private Service Connect NAT subnet.If there is a performance issue like network latency or packet drops, check if Live Data Plane Analysis is available to determine a baseline and isolate an issue with the application or service. Also, check the Metrics Explorer for any connections or port exhaustion and packet drops.

Working with Cloud Support

Once that you have pinpointed the issue and you have analyzed the problem, you may need to reach out to Cloud Support for further assistance. To facilitate a smooth experience, be sure to explain your needs, clearly describe the business impact and give enough context with all the information collected.

The sample app is available in github:

Start the dockerized application:

This should insert records into Cloud SQL PostgreSQL. After verification, stop the application:

4. Configure and migrate schema

In the “Configure Schema” view, clicking on each table shows the schema for PostgreSQL and Spanner in a side-by-side comparison.

We then start it:

After verifying that the application has inserted a few rows successfully, we can stop it and delete the sample data.

6. Minimal downtime migration

SMT orchestrates the “Minimal Downtime Migration” process by:

Loading initial data from the source database into the destination.Applying a stream of change data capture (CDC) events.

SMT generates useful links for monitoring the migration.

7. Using Spanner

Stop the application that is using PostgreSQL:

Wait for the Spanner to catch up, once Dataflow’s backlog reaches zero, switch to the Spanner application:

With that we successfully completed our demo app migration using SMT!

Now you can click on “End Migration” and clean up the jobs.

Conclusion

Through careful planning and utilizing SMT, we can minimize downtime during the migration to Spanner, ensuring an efficient transition.

Imagen 2 on Vertex AI brings Google’s state-of-the-art generative AI capabilities to application developers. With Imagen 2 on Vertex AI, application developers can build next-generation AI products that transform their user’s imagination into high-quality visual assets, in seconds.

With Imagen, you can:

Generate novel images using only a text prompt (text-to-image generation).Edit an entire uploaded or generated image with a text prompt.Edit only parts of an uploaded or generated image using a mask area you define.Upscale existing, generated, or edited images.Fine-tune a model with a specific subject (for example, a specific handbag or shoe) for image generation.Get text descriptions of images with visual captioning.Get answers to a question about an image with Visual Question Answering (VQA).

In this blog post, we will show you how to use the Vertex AI SDK to access Imagen 2’s image generation feature. This blog is designed to be a code-along tutorial, so you can code along as you read.

Environment setup

Before you get started, please ensure that you have satisfied the following criteria:

Have an existing Google Cloud Project or create a new one.Set up a billing account for the Google Cloud project,Enable the Vertex AI API on the Google Cloud project.

To use Imagen 2 through the SDK, run the following command in the preferred Python environment to install the Vertex AI SDK:

initialize the image generation model using the following code:

You should see a generated image which is similar to the image shown below:

If you prefer to save the generated images, you can execute the following code:

Head over to the Imagen documentation and the API reference page to learn more about Imagen 2’s features. Additionally, the concepts presented in this blog post can be accessed in the form of a Jupyter Notebook in our Generative AI GitHub repository, providing an interactive environment for further exploration. To keep up with our latest AI news, be sure to check out the newest Gemini models available in Vertex AI and Gemma, Google’s new family of lightweight, state-of-the-art open models.

From the front lines of cyber investigations, the Mandiant Managed Defense team observes how cybercriminals find ways to avoid defenses, exploit weaknesses, and stay under the radar. We work around the clock to develop new techniques to identify attacks and stop breaches for our clients.

Our aim is to help organizations better protect themselves against the security challenges of today, as well as the future. To minimize the impact of an incident, organizations must prioritize a robust detection posture and a well-structured response plan. This blog highlights our key observations from the many engagements we were involved with in 2023.

Infostealers: 2023’s most prevalent malware

Infostealer, information stealing malware, was the most prevalent type of malware observed by Managed Defense in 2023. The trend coincides with the observed demand and supply of stolen data within underground forums and markets. Infostealers are lucrative for cybercriminals, as stolen data can be used for extortion or to facilitate advancing multiple stages of an intrusion.

For example, infostealers can obtain credentials for initial access or lateral movement, and they also gather information about an environment. This reconnaissance data can be used to tailor future attacks, significantly escalating a breach’s severity and impact.

Infostealers are often distributed through cracked or pirated software as a lure. Cybercriminals used social engineering and search engine optimization (SEO) tactics to more effectively distribute their malware. In July, Managed Defense uncovered a campaign that used fake Zotero and Notion installers (free, open-source reference management tools) to gain unauthorized access to targeted machines. These thematically-relevant lures demonstrate how easily cybercriminals can tailor their attacks to specific industries and organizations.

The increasing use of Wiper malware alongside today’s ransomware threat, organizations are at greater risk of losing their data. We recommend organizations implement a robust data protection strategy.

Generative AI and the adversary

The prevalence of malware distribution through social media, scam calls, and email phishing schemes continues unabated. These methods are designed to lure users into downloading malware or unknowingly surrendering their credentials on phishing pages, and they remain a persistent and concerning reality that can result in monetary loss for both businesses and individuals.

Threat actors like UNC3944 (also known as Scattered Spider) have mastered these tactics, using phone-based social engineering and smishing to infiltrate victim organizations, exfiltrate sensitive data, and extort victims with ransomware. We have published a guide to how to mitigate threats from this specific threat group.

Other threat groups also use social engineering to further their goals, including:

FIN6 and UNC4962 have employed lures such as resume-themed files and fake job postings on fabricated personal websites to lead users into downloading malicious software.The BazarCall campaign initiated by UNC3753 stands out because it used phone calls to fake businesses as part of its phishing scheme, ultimately tricking victims into downloading remote access tools that grant the attackers an initial foothold.UNC4214 has been using social media and messaging platforms, and malicious social media advertisements, to distribute credential-stealing malware. Their ultimate goal is to infiltrate organizations, gain control of social media business accounts, and then use those accounts for financial gain.

As threat actors regularly evolve their tactics to use new and emerging technologies, we anticipate that cybercriminals will integrate AI more deeply into their toolkits. Realistic, AI-generated content can enhance the persuasiveness of these social media lures, making them more difficult to identify and avoid. However, while there is threat actor interest in this technology, adoption has been limited so far.

Organizations should explore the implementation of security frameworks like the Principle of Least Privilege (PoLP) and Zero Trust security. PoLP centers on granting user access to only what is strictly necessary, while Zero Trust adds a focus on continuous verification and access control. Organizations also may benefit from investing in a strong detection and response program, underpinned by modern Security Operations tools such as Chronicle. An organization’s ability to demonstrate resilience through rapid and effective incident response will foster community confidence and build trust in its products and services.

Looking ahead and next steps

The Managed Defense team is committed to continuous innovation that empowers our customers. This commitment drives our ongoing development of robust, helpful solutions, such as the recently-launched Mandiant Hunt for Chronicle, which can provide direct proactive threat hunting directly on Chronicle data. Our commitment also motivates our partnerships with industry leaders including Microsoft Defender for Endpoint, CrowdStrike, and SentinelOne, so we can offer flexibility and choice within your security stack.

We are excited about the future, reimagining security operations to find new ways to relentlessly protect our customers from impactful cyberattacks.

Generative AI has become the number one technology of interest across many industries over the past year. Here at Google Cloud for Games, we think that online game use cases have some of the highest potential for generative AI, giving creators the power to build more dynamic games, monetize their games better, and get to market faster. As part of this, we’ve explored ways that games companies can train, deploy, and maintain GenAI utilizing Google Cloud. We’d like to walk you through what we’ve been working on, and how you can start using it in your game today. While we’ll focus on gen AI applications, the framework we’ll be discussing has been developed with all machine learning in mind, not just the generative varieties.

Long term, the possibilities of gen AI in Games are endless, but in the near term, we believe the following are the most realistic and valuable to the industry over the next 1-2 years.

Game productionAdaptive gameplayIn-game advertising

Each of these helps with a core part of the game development and publishing process. Generative AI in game production, mainly in the development of 2D textures, 3D assets, and code, can help decrease the effort to create a new game, decrease the time to market, and help make game developers more effective overall. Thinking towards sustaining player engagement and monetizing existing titles, ideas like adaptive dialogue and gameplay can keep players engaged, and custom in-game objects can keep them enticed. In-game advertising opens a new realm of monetization, and allows us not only the ability to hyper-personalize ads to views, but to personalize their placement and integration into the game, creating seamless ad experiences that optimize views and engagement. If you think about the time to produce a small game, never mind a AAA blockbuster, development of individual game assets consumes an immense amount of time. If generative models can help reduce developer toil and increase the productivity of studio development teams by even a fraction, it could represent a faster time to market and better games for us all.

As part of this post, we introduce our Generative AI Framework for Games, which provides templates for running gen AI for games on Google Cloud, as well as a framework for data ingest and storage to support these live models. We walk you through a demo of this framework below, where we specifically show two cases around image generation and code generation in a sample game environment.

But before we jump into what we’re doing here at Google Cloud, let’s first tackle a common misconception about machine learning in games.

Cloud-based ML plus live services games are a go

It’s a common refrain that running machine learning in the cloud for live game services is either cost prohibitive or prohibitive in terms of the induced latency that the end user experiences. Live games have always run on a client-server paradigm, and it’s often preferable that compute-intensive processes that don’t need to be authoritative run on the client. While this is a great deployment pattern for some models and processes, it’s not the only one. Cloud-based gen AI, or really any form of AI/ML, is not only possible, but can result in significantly decreased toil for developers, and reduced maintenance costs for publishers, all while supporting the latencies needed for today’s live games. It’s also safer — cloud-based AI safeguards your models from attacks, manipulation, and fraud.

Depending on your studio’s setup, Google Cloud can support complete in-cloud or hybrid deployments of generative models for adaptive game worlds. Generally, we recommend two approaches depending on your technology stack and needs;

If starting from scratch, we recommend utilizing Vertex AI’s Private Endpoints for low latency serving, which can work whether you are looking for a low ops solution, or are running a service that does not interact with a live game environment.If running game servers on Google Cloud, especially if they are on Google Kubernetes Engine (GKE), and are looking to utilize that environment for ultra-low latency serving, we recommend deploying your models on GKE alongside your game server.

Let’s start with Vertex AI. Vertex AI supports both public and private endpoints, although for games, we generally recommend utilizing Private Endpoints to achieve the appropriate latencies. Vertex AI models utilize what we call an adaptor layer, which has two advantages: you don’t need to call the entire model when making a prediction, and any fine tuning conducted by you, the developer, is contained in your tenant. Compared to running a model yourself, whether in the cloud or on prem, this negates the need to handle enormous base models and the relevant serving and storage infrastructure to support them. As mentioned, we’ll show both of these in the demo below.

If you’re already running game servers on GKE, you can gain a lot of benefit from running both proprietary and open-source machine learning models on GKE as well as taking advantage of GKE’s native networking. With GKE Autopilot, our tests indicate that you can achieve prediction performance in the sub-ms range when deployed alongside your game servers. Over the public internet, we’ve achieved low millisecond latencies that are consistent, if not better, with what we have seen in classic client side deployments. If you’re afraid of the potential cost implications of running on GKE, think again — the vast majority of gaming customers see cost savings from deploying on GKE, alongside a roughly 30% increase in developer productivity. If you manage both your machine learning deployments and your game servers with GKE Autopilot, there’s also a significant reduction in operational burden. In our testing, we’ve found that whether you are deploying models on Vertex or GKE, the cost is roughly comparable.

Unified data platforms enable real-time AI

AI/ML driven personalization thrives on large amounts of data regarding player preferences, gameplay, and the game’s world and lore. As part of our efforts in gen AI in games, we’ve developed a data pipeline and database template that utilizes the best of Google Cloud to ensure consistency and availability.

Live games require strong consistency, and models,whether generative or not, require the most up-to-date information about a player and their habits. Periodic retraining is necessary to keep models fresh and safe, and globally available databases like Spanner and BigQuery ensure that the data being fed into models, generative or otherwise, is kept fresh and secure. In many current games, users are fragmented by maps/realms, with hard lines between them, keeping experiences bounded by firm decisions and actions. As games move towards models where users inhabit singular realms, these games will require a single, globally available data store. In-game personalization also requires the live status of player activity. A strong data pipeline and data footprint is just as important for running machine learning models in a liveops environment as the models themselves. Considering the complexity of frequent model updates across a self-managed data center footprint, we maintain it’s a lighter lift to manage the training, deployment, and overall maintenance of models in the cloud.

By combining a real-time data pipeline with generative models, we can also inform model prompts about player preferences, or combine them with other models that track where, when, and why to personalize the game state. In terms of what is available today, this could be anything from pre-generated 3D meshes that are relevant to the user, retexturing meshes to different colors, patterns or lighting to match player preferences or mood, or even the giving the player the ability to fully customize the game environment based natural language. All of this is in service of keeping our players happy and engaged with the game.

Demoing capabilities

Let’s jump into the framework. For the demo, we’ll be focusing on how Google Cloud’s data, AI, and compute technology can come together to provide real-time personalization of the game state.

The framework includes:

Unity for the client and server

Open source:

TerraformAgones

Google Cloud:

GKEVertex AIPub/SubDataflowSpannerBigQuery

As part of this framework, we created an open-world demo game in Unity that uses assets from the Unity store. We designed this to be an open world game — one where the player needs to interact with NPCs and is guided through dynamic billboards that assist the player in achieving the game objective.. This game is running on GKE with Agones, and is designed to support multiple players. For simplicity, we focus on one player and their actions.

Now, back to the framework. Our back-end Spanner database contains information on the player and their past actions. We also have data on their purchasing habits across this make-believe game universe, with a connection to the Google Marketing Platform. This allows us in our demo game to start collecting universal player data across platforms. Spanner is our transactional database, and BigQuery is our analytical database, and data flows freely between them.

As part of this framework, we trained recommendation models in Vertex AI utilizing everything we know about the player, so that we can personalize in-game offers and advertising. For the sake of this demo, we’ll forget about those models for a moment, and focus on two generative AI use cases: image generation, NPC chat, and code generation for our adaptive gameplay use case. To show you both deployment patterns that we recommend for games, deploying on GKE alongside the game server, and utilizing Vertex AI. For image generation, we host an open-source Stable Diffusion model on GKE, and for code generation and NPC chat we’re using the gemini-pro model within Vertex AI. In cases where textures need to be modified or game objects are repositioned, we are using the Gemini LLM to generate code that can render, position, and configure prefabs within the game environment.

As the character walks through the game, we adaptively show images to suggest potential next moves and paths for the player. In practice, these could be game-themed images or even advertisements. In our case, we display images that suggest what the player should be looking for to progress game play.

In the example above, the player is shown a man surrounded by books, which provides a hint to the player that maybe they need to find a library as their next objective. That hint also aligns with the riddle that the NPC shared earlier in the game. If a player interacts with one of these billboards, which may mean moving closer to it or even viewing the billboard for a preset time, then the storyline of our game adapts to that context.

We can also load and configure prefabs on the fly with code generation. Below, you’ll see our environment as is, and we ask the NPC to change the bus color to yellow, which dynamically updates the bus color and texture.

Once we make the request, either by text or speech, Google Cloud GenAI models generate the exact code needed to update the prefab in the environment, and then renders it live in the game.

While this example shows how code generation can be used in-game, game developers can also use a similar process to place and configure game objects within their game environment to speed up game development.

If you would like to take the next step and check out the technology, then we encourage you to explore the Github link and resources below.

Additionally, we understand that not everyone will be interested in every facet of the framework. That’s why we’ve made it flexible – whether you want to dive into the entire project or just work with specific parts of the code to understand how we implemented a certain feature, the choice is yours.

If you’re looking to deepen your understanding of Google Cloud generative AI, check out this curated set of resources that can help:

Generative AI on Google CloudGetting started with generative AI on Vertex

Last but not least, if you’re interested in working with the project or would like to contribute to it, feel free to explore the code on Github, which focuses on the GenAI services used as part of this demo:

Generative AI Quickstart for Gaming

Monitoring microservices in the cloud has become an increasingly cumbersome exercise for teams struggling to keep pace with developers’ rapid application release velocity. One way to make things easier for overloaded security teams is to use the open-source runtime security platform Falco to quickly identify suspicious behavior in Linux containers and applications. The overarching goal of Falco is to uncomplicate visibility in rapidly-deployed, cloud-first workloads everywhere, making life less stressful for your engineering organization.

Falco can help with other vital security tasks, too. You can use it with Google Kubernetes Engine (GKE), for example, to monitor cluster and container workload security. Like a Swiss army knife, Falco can detect abnormal behavior in all types of cloud-first workloads. Its plugin architecture can be used to monitor almost anything that emits a data stream in a known format, including a wide array of Google Cloud services.

Let’s examine how Falco and the gcpaudit plugin enables runtime security monitoring across a wide range of Google Cloud services.

Before we go much further, it’s important to understand the plugin model. Falco’s plugins are shared libraries that conform to a documented API, hooking into Falco’s core functionality. You can use this model to add new event sources, define new fields to be extracted from events, parse content extracted from a data stream, and inject events asynchronously in a given data stream. Officially maintained Falco plugins can be found in its plugin registry.

How the gcpaudit plugin can help analyze log data

The gcpaudit plugin ingests audit logs for several key Google Cloud services, including:

Compute EngineKMSCloud Armor WAFIAMCloud StorageBigQueryCloud SQLPub/SubCloud LoggingCloud Functions

This plugin uses an optimized Google Cloud logging filter or “sink” to send the most critical events from a monitored service to a Pub/Sub topic running within an organization’s project. The gcpaudit plugin acquires events from thePub/Sub topic, applies filters and enrichment (using custom metadata) which operators can then access in Falco.

This filtering process is based on a set of custom Falco rules authored with the MITRE ATT&CK framework in mind. For example, when an action, such as deleting a virtual machine (VM) or removing a database backup, triggers one of these rules, Falco sends out an alert. The plugin provides a centralized view of security events across multiple Google Cloud services and can help detect unauthorized access, data exfiltration, and other types of malicious activity.

To support your other security operations processes, such as building playbooks, event correlation, and triaging incidents, these alerts can then be sent to a security information and event management (SIEM) system such as Chronicle Security Operations. Falco sends alerts to stdout, syslog, an HTTP/HTTPS client or gRPC endpoint by default, but you can leverage a companion project, FalcoSidekick, to send alerts to other destinations in your environment.

There are some important factors to consider with Falco. First, you can’t use Falco on GKE clusters running in Autopilot mode, as it requires privileged access to install its driver. To avoid duplication, only one instance of the gcpaudit plugin should be deployed per monitored environment. For example, you wouldn’t install the plugin on each node in a GKE cluster. Cloud Logging aggregates alerts for a set of monitored services, then forwards to the plugin. The plugin can be installed directly into a compute instance VM, or it can be run as a pod in GKE.

With over 60 outputs such as messaging applications, observability tools, object stores, and databases, you have many options. Since Falco’s an open source project, you can contribute your own outputs, too.

Microservices can be complex, and it can be difficult to ensure alerts get to the right team early enough to have a meaningful impact. Getting the necessary visibility required to secure them can be a challenge, especially when using Kubernetes. By taking advantage of the gcpaudit plugin, security teams can more quickly identify events and respond to incidents in these Google Cloud workloads and improve their organization’s security posture.

To get started and help make your applications “secure every second,” you can try out Falco’s gcpaudit plugin with this detailed walkthrough.

The next generation of Autocomplete is now available in Preview, offering seamless integration with Address Validation, more intuitive pricing, and support for expanded place types from the new Places API. It’s available to all developers at no cost during Preview. Autocomplete builds on our recent launch of the new Places API, which offers new features in Text Search, Place Details and Photos, and Nearby Search, to make it easier for you to surface helpful information about the world in the products you build.

What sets apart Google’s Autocomplete

With Autocomplete, you can help your users quickly find the exact place they’re looking for by automatically suggesting businesses and points of interest as they type. Autocomplete utilizes the speed of Google Search to quickly search and suggest addresses from one of the most accurate and comprehensive models of the world, with over 50 million daily updates and coverage in more than 250 countries and territories. In addition to our extensive address data, Autocomplete helps you bring rich location data for over 250 million places to your users, including opening hours, price levels, user ratings, and much more.

Improvements to the next generation Autocomplete

Seamless integration with Address Validation

Global ecommerce this year will reach almost $6 trillion dollars in sales.1 But ecommerce growth can be challenged, in part due to complications with checkout. In fact, 70% of all online shopping carts are abandoned2. There are also challenges with fulfillment. According to a recent survey, 71% of surveyed shoppers have contacted customer service due to shipping and/or delivery issues regarding online orders3 . To address this, we’ve created a new Autocomplete with Address Validation product that helps you decrease cart abandonment, increase customer repeat rate, and delight customers with faster checkout. It also helps you drive operational efficiencies by catching and fixing address errors earlier in the process to make faster deliveries, save costs, and avoid customer churn due to incorrect deliveries. At a CPM of $25 per 1,000 sessions, this product also provides significant cost savings when compared to using the new Autocomplete and Address Validation APIs separately, which is a CPM of $32 per 1,000 sessions.

Supports expanded place types from new Places API

The next generation Autocomplete supports all of the expanded place types from the new Places API. Places API now provides even more details and place types, which means your users will be able to find detailed information about places of interest by type, such as best sushi or nearby hiking areas. With the new Places API, we’ve nearly doubled the number of supported place types since the previous version. Now there are nearly 200 place types including coffee shops, playgrounds, EV charging stations, and more. You’ll also be able to provide updated accessibility information for places, including wheelchair-accessible seating, restrooms, and parking.

Simpler and more intuitive pricing

We’ve changed our pricing to make it simpler, more intuitive, and better match how your users use Autocomplete. For example, sometimes your users don’t always finish their checkout, or they can select a pickup point and they accidentally close their browser. With the new Autocomplete session-based pricing, if your user doesn’t complete their session (e.g. they enter a single character in your search box and then close their browser), you will only be charged for an Autocomplete request instead of a full session with Address Validation or Place Details.

We now offer three Autocomplete SKUs that line up to the most common use cases

Checkout and delivery–the Autocomplete with Address Validation product is designed for e-commerce use cases. It helps you provide a seamless checkout and frictionless fulfillment experience, helping drive more revenue and an enhanced customer experience.Discover new places–the Autocomplete with Place Details Preferred product is ideal for when you want to help users not only find places, but discover more about those places with rich place details–such as opening hours, user ratings and reviews, and price levels–to make decisions about where to go.Find the exact place a user is looking for–the Autocomplete with Place Details Location product can be used to help users quickly find the exact place they’re looking for by automatically suggesting addresses, businesses and points of interest as they type, and then showing the chosen location on a Google map.

New Google Cloud service infrastructure

Like the other products offered with the new Places API, the new Autocomplete offers modern security settings with OAuth-based authentication and is built on Google Cloud’s service infrastructure to give you even more peace of mind as you build with us. This provides an alternative to API key-based authentication of requests to the new Places API, which are still supported.

Get started with next generation Autocomplete

Try out the next generation Autocomplete at no cost during the Preview phase. To learn more, check out the documentation and the updated pricing. To learn more about the new Places API, check out our webpage and documentation.

To get started with the next generation Autocomplete, ensure “Places API (New)” is enabled on your Google Cloud project. If you are using the Place Autocomplete web service, follow the link to the docs to learn how to update your API call to the next generation of Autocomplete. If you are calling Place Autocomplete via our Maps JavaScript API, or Android, or iOS SDKs, we will be launching the next generation Autocomplete on these platforms in the coming months.

We’d love to see what you’re creating with the next generation Autocomplete and Places API, so be sure to tag us at @GMapsPlatform on X and #GoogleMapsPlatform on YouTube and LinkedIn. We can’t wait to see what you build.

Sources

eMarketer, March 2023Baymard Institute, 2023UPS Capital, April 2023

Run the workflow with some countries:

You’ll get the results as fast as the slowest LLM call. Much faster than making each call sequentially. In a few seconds, you should see the output map with countries and their histories:

Second, Gemini has a streaming endpoint, which means responses come in chunks and you need to combine the text in each chunk to get the full text. That’s why we have the following steps to extract and combine text from each chunk:

The full sample is in our GitHub repository here.

Call VertexAI Gemini Pro Vision from Workflows to describe an image

The real power of Gemini is its multimodal nature, which means it can generalize and understand and operate across different types of information such as text, code, audio, image and video.

So far, we’ve been generating text. Can we use Workflows to take advantage of the multimodal nature of Gemini? Sure, we can. As an example, you can use Workflows to get a description of this image from Gemini:

Run the workflow:

You should see an output similar to the following:

Nice! The full sample is in our GitHub repository here. As an exercise, you can even extend this sample to describe a number of images in parallel and save the results to txt files back to the Cloud Storage bucket.

Summary

There are many ways of calling LLMs with client libraries, generated libraries, REST APIs, LangChain. In this post, I showed you how to call some of the gen AI models from Workflows. With its parallel steps and retry steps, Workflows offers a robust way of calling gen AI models. With its Eventarc integration, Workflows allows you to have event-driven LLM applications.

If you want to learn more, check our Access Vertex AI models from a workflow documentation page. As always, if you have any questions or feedback, feel free to reach out to me on Twitter @meteatamel.

As new digital-native exchanges explore running exchanges in the cloud, the focus is often on specific requirements such as latency, determinism, and resilience. Generalized storage and compute paradigms are not enough — supporting trading in the cloud requires a unique approach. This is why, for several years, Google Cloud has adopted workload optimization and intentional design as central principles for our infrastructure platform. We engineer golden paths from silicon to the customer workload, using a combination of purpose-built infrastructure, prescriptive architectures, and an open ecosystem to deliver workload-optimized infrastructure1.

An example of how we offload workloads from a host is Titanium, a system of tailored, custom silicon and multiple tiers of scale-out offloads that improve the performance, reliability, and security of our customers’ workloads. In particular, we’ve seen how a combination of custom infrastructure, prescriptive architectures, and an open ecosystem to deliver workload-optimized infrastructure can benefit latency-sensitive industry workloads such as trading exchanges.

Ensuring that digital-native exchanges can operate in the cloud is about providing capabilities that can help ensure performant, scalable, and resilient markets for the future. In particular, many exchanges rely on Aeron, an OSS based low-latency, high-throughput, fault-tolerant messaging framework. As outlined in a recent Aeron Performance Testing on Google Cloud by Adaptive, a capital markets consultancy, software provider and Google Cloud partner, running Aeron on Google Cloud’s purpose-built infrastructure can help directly address the latency, scale and resilience required to operate a cloud-based exchange, market maker, and market data aggregator.

Performance to support cloud exchange needs

The Aeron messaging fabric leveraging Google Cloud Compute Engine C3 virtual machine instance types with Data Plane Development Kit (DPDK) was able to deliver node-to-node communication of 100,000 messages per second of 288-byte data packets (e.g. FIX order message) at 18 microseconds P99 (µs – 1 millionth of a second). These metrics (Figure 1) demonstrate the ability to run with low-latency infrastructure, enabling market participants to trade on cloud exchanges with confidence. More importantly, the configuration achieved deterministic repeatability, high throughput, low latency, and high availability via Aeron Cluster with an the additional latency cost of just 18µs at P99.

Figure 1

Addressing raw latency and jitter is important, but the ability to meet growth and market volatility is more important for rapidly growing digital exchanges and their participants. Google Cloud’s design and investments in its on-host offloading and accelerator include the Infrastructure Processing Unit (IPU), which demonstrated processing 4.7 million messages per second at 288 byte size to distribute over 10 Gb per second in a single thread (Figure 2). This message rate far exceeds most exchange requirements for match-engine communication and data distribution. The Options Price Reporting Authority (OPRA) is possibly the only data feed constantly operating beyond these message requirements.

Figure 2

While exchanges running on-prem are in control of their infrastructure, that can present challenges. There are multiple single points of failure (network, hardware, hardware supply chain, software, and people)and manual failover operations, while traditional high availability solutions remain complex and error-prone. The Aeron Cluster and Google Cloud approach provides automated failover, reducing infrastructure risk and the potential for operator error. Finally, firms requiring specific resilience patterns can automate zone separation using an organizational policy constraint that helps ensure that node placement (organization, folder, or project) in the resource hierarchy respects their resiliency requirements.

To help enable resilience on a hardware level, Google Cloud’s C3 instance types support Advanced Maintenance, which can allow Google Cloud to coordinate with customers on updating software and firmware without disrupting the customer’s workloads. Additionally, C3 Advanced Maintenance makes it possible to postpone maintenance on instances running critical workloads for up to one month, and more importantly, notifies you a week in advance of maintenance required on a given instance. By combining Aeron software and Google Cloud’s C3 capabilities, exchanges can gracefully manage their underlying infrastructure changes regardless of the source of change. This includes hot upgrades, enabling new features, and adding nodes, all without incurring service interruption.

Maintenance notifications are API-driven and provide details of every planned maintenance event, such as the window schedule and status. Aeron and exchange applications can also natively handle maintenance notifications, enabling exchanges to automatically establish operating models that are aligned to their specific operational requirements, for example low-volume hours, established change windows, and regulatory needs.

What’s next?

Exchanges pride themselves on being critical locations for global capital markets with which to manage risk. They are especially important in times of volatility from macro events, institutional failures, and natural disasters. To ensure that digital-native exchanges are available when the global financial community needs them, they need the right mix of performance, scale, and reliability.

Google Cloud’s intentional design philosophy helps ensure that exchanges are positioned to meet these requirements today, and more importantly, into the future. Exchanges working with Google Cloud can benefit from our leadership and investments including work on next-generation network protocols such as Falcon, a reliable low-latency hardware transport that contributed to the Association for Computing Machinery and Internet Engineering Task Force. Exchanges will also continue to see industry-specific performance improvements via our work with the open-source low-latency Aeron messaging framework via our partnerships with Adaptive.

If you want to run Aeron performance tests in your own environment, you can request the Aeron Performance Testing guide. The Aeron team also has a set of infrastructure provisioning modules that help with the setup and deployment of the benchmark tests. Please get in touch with the Aeron team for help with test setup on Google Cloud. And as always, don’t hesitate to reach out to your Google Cloud team with any questions you may have.

1. Titanium underpins Google’s workload-optimized infrastructure | Google Cloud Blog. Cluster with an the additional latency cost of just 18µs at P99.
2. Titanium: A robust foundation for workload-optimized cloud computing

Today, Google announced Gemma, a family of lightweight, state-of-the art open models built from the same research and technology that we used to create the Gemini models. We’re pleased to share that Google Cloud customers can get started today customizing and building with Gemma models in Vertex AI and running them on Google Kubernetes Engine (GKE). The release of Gemma and our updated platform capabilities are the next phase of our commitment to making AI more open and accessible to developers on Google Cloud.

Gemma is now available on Google Cloud

Gemma models share technical and infrastructure components with our capable Gemini models. This enables Gemma models to achieve best-in-class performance for their sizes compared to other open models. We are releasing weights in two sizes: Gemma 2B and Gemma 7B. Each size is released with pre-trained and instruction-tuned variants to enable both research and development.

Gemma supports tools that Google Cloud developers love and use today, including Colab and Kaggle notebooks, as well as frameworks like JAX, PyTorch, Keras 3.0, and Hugging Face Transformers. Gemma models can run on a laptop, workstation, or on Google Cloud. With these new open models, developers can now build with and customize in Vertex AI and run it on GKE. To maximize industry-leading performance, we have collaborated with NVIDIA to optimize Gemma for NVIDIA GPUs.

Unlocking the power of Gemma in Vertex AI

Gemma joins over 130 models in Vertex AI Model Garden, including our recently announced expanded access to Gemini: Gemini 1.0 Pro, 1.0 Ultra, and 1.5 Pro models.

By using Gemma models on Vertex AI, developers can take advantage of an end-to-end ML platform that makes tuning, managing, and monitoring models simple and intuitive. With Vertex AI, builders can reduce operational overhead and focus on creating bespoke versions of Gemma that are optimized for their use case. For example, using Gemma models on Vertex AI, developers can:

Build generative AI apps for lightweight tasks such as text generation, summarization, and Q&AEnable research and development using lightweight-but-customized models for exploration and experimentationSupport real-time generative AI use cases that require low latency, such as streaming text

Vertex AI makes it easy for developers to turn their own tuned models into scalable endpoints that can power AI applications of all sizes.

Scale from prototype to production with Gemma on GKE

GKE provides tools to build custom apps, from prototyping simple projects to rolling them out at enterprise scale. Today, developers can also deploy Gemma directly on GKE to create their own gen AI apps for building prototypes or testing model capabilities:

Deploy custom, fine-tuned models in portable containers alongside applications using familiar toolchainsCustomize model serving and infrastructure configurations without the need to provision or maintain nodesIntegrate AI Infrastructure fast with the ability to scale to meet the most demanding training and inference scenarios

GKE delivers efficient resource management, consistent ops environments, and autoscaling. In addition, it helps enhance these environments with easy orchestration of Google Cloud AI accelerators, including GPUs and TPUs, for faster training and inference when building generative AI models.

Get started with Gemma on Google Cloud today

You can start working with Gemma models today on Google Cloud in Vertex AI and GKE. For more information about Gemma, access quickstart guides on ai.google.dev/gemma.

Are you ready to bridge the digital divide, deliver citizen-centric services, and future-proof your agency in 2024? Public sector technology is evolving at breakneck speed, with AI, cybersecurity, and sustainable solutions taking center stage. Join us at Google Cloud Next ‘24, April 9-11. Here are 10 great reasons to register now for Next ‘24!

Master cutting-edge AI: Dive into Google’s advanced AI tools through workshops, demos, and sessions that propel your mission forward. See real-world examples of AI transforming areas like disaster relief and research.Supercharge your skills: Gain hands-on experience with the latest cloud technologies and data-driven solutions. Engage your leadership with your newfound expertise and elevate your career.Network with your public sector community and cross industry experts: Connect with fellow leaders, Google Cloud experts, and industry peers and partners. Share best practices, exchange ideas, and build collaborations that drive innovation.Learn from inspiring success stories: Discover impactful case studies, like empowering examiners with AI, building disaster resilience, and revolutionizing education. See how technology is making a difference and get inspired to do the same.Deep dive into your interests: Explore 500+ sessions across 10 tracks, covering security, analytics, citizen engagement, and more. Learn from 30+ public sector speakers, including leaders from the states of New Jersey and Wisconsin, and Deloitte Consulting to gain industry and real-world insights.Get ahead of cybersecurity threats: Discover how AI-powered security solutions are protecting public sector agencies. Gain the knowledge and tools to confidently safeguard your data and infrastructure.Embrace sustainable solutions: Learn about technologies that promote environmental responsibility and social good. Be part of building a better future for everyone.Save big with special discounts: Enjoy exclusive pricing for government and education registrants. Use your business email at registration. Make your budget stretch further and invest in your professional development.Reimagine your mission: See how technology can transform public services and create a brighter future for your community. Leave feeling energized and ready to make a real impact.Join a movement for good: Connect with a vibrant community of passionate public servants committed to positive change. Together, we can harness technology to serve better, faster, and stronger.

Ready to make a difference? Register for Google Cloud Next ’24 and unlock your full potential and help take your mission to the next level!

The potential of video analytics and generative AI to revolutionize industries is immense. These technologies are opening new frontiers in automated insights, decision-making, and content generation. By marrying AI insight and audio data, organizations are realizing benefits across the span of business, from increased sales and revenue to enhanced customer experiences and reduced costs. Any organization not exploring these tools risks falling behind the competition.

This post offers a glimpse into the tremendous potential at the intersection of visual and linguistic intelligence. It reveals how we can harness video analytics and AI to unlock game-changing insights, create bespoke buyer experiences, and elevate customer relationships. The possibilities stretch as far as the imagination; and the time to start this exploration is now.

The demand for advanced video analysis and content generation is skyrocketing in today’s digital age. In fact, per Precedence & Straits research, the video analytics industry is expected to reach $50.7 billion by 2032, while the gen AI industry is expected to reach $118.06 billion by 2032. It’s clear that organizations are searching for AI solutions to extract insights from audio transcripts within videos and streamline processes through automation.

Combining the power of analyzing video content at scale and leveraging large language models to generate contextual narratives allows organizations to tap into a new realm of possibilities, including:

Sports Analytics: Leverage gen AI to transform post-game interviews into blog articles. These insights enable coaches to optimize training, strategize more effectively, and spot star talent.

Real Estate Retail: Analyze property walkthrough videos with computer vision, extracting critical details about layout, condition, and amenities. Feed transcripts into generative models to effortlessly construct detailed listing descriptions. AI can then analyze and compare against similar listings to accurately estimate market value.

Retail: Ingredients extracted in real-time from cooking show transcripts. As the chef talks through the recipe, matching items automatically land in your cart. Streamlining the shopping experience for aspiring home cooks.

The Video Intelligence API from Google Cloud and gen AI on Google Cloud offer complimentary video analysis and content creation capabilities. The Video Intelligence API enables businesses to efficiently analyze video content at scale, with features for labeling, speech transcription, shot detection, and more. Meanwhile, gen AI allows for the automated generation and summarization of text, images, audio, and video. Together, these technologies provide a comprehensive solution – the Video Intelligence API extracts vital insights from video assets. At the same time, gen AI leverages those insights to create novel experiences like chatbots, listings, and articles. This end-to-end pipeline from analysis to content creation delivers tremendous value.

Exploring the Synergy:

Imagine a real estate company using the Video Intelligence API to analyze video recordings of home tours performed by a real estate retail agent. The API first analyzes home tour footage, identifying property features and conditions. Transcripts are then fed into a generative model which produces custom listing descriptions for each home. By pairing advanced video analytics with AI-generated copy, properties can be indexed with rich detail – including visual details like layout and amenities and customized descriptive text for each listing. The end result is enhanced exposure on real estate search platforms and a better experience for potential buyers.

The potential applications of video analytics paired with gen AI extend far beyond real estate listings. This powerful combo delivers value across sectors:

Personalized home search: Leverage multi-modal analytics to personalize the home search for potential buyers. By utilizing Google’s foundation models, organizations can generate text descriptions of homes that match a buyer’s specific criteria. These personalized descriptions can then curate a customized list of homes that align with the buyer’s preferences, significantly enhancing their search experience.Search powered by gen AI: Video analytics and Chirp, Google Cloud’s speech model, can be leveraged to automatically transcribe video walkthroughs of homes. This transcription makes it easier for customers to search for specific features or details within the videos. For instance, customers can search for homes with “hardwood floors” or “granite countertops” and quickly find videos mentioning these features. Now, customers are empowered to find their ideal homes more efficiently and effectively.Customer insights: The combination of Video Intelligence API and gen AI allows organizations to analyze customer feedback from home walkthroughs. By extracting insights from this feedback, businesses gain a deeper understanding of customers’ likes, dislikes, and preferences. This valuable information can be used to personalize marketing and sales efforts, optimize the customer experience, and even enhance virtual reality walkthroughs for customers who are unable to visit a property physically.Question and Answer (QnA) on house features based on video summaries: The Video Intelligence API can summarize the key features of a property from home tour videos. By integrating this summary with gen AI-powered question-answering capabilities, organizations can create an interactive QnA system. Potential buyers can ask questions about the property’s features, condition, or any other aspect, and the system responds with accurate and informative answers based on the video summary. This enhances the customer experience by providing instant access to relevant information and streamlining the decision-making process.

The combinations of the Video Intelligence API and gen AI unlock endless possibilities and together, their analytical and creative strengths pave the way for innovation across sectors. In real estate, this fusion delivers tangible benefits:

Increased Sales: Realtors, freed from manual tasks, can invest more time building client relationships and closing deals.Enhanced Customer Experience: With greater availability, realtors can provide personalized guidance tailored to each client.Cost Savings: By automating rote tasks, firms reduce expenses and labor costs.

Architecture:

Sample Video used for this demo can be found here

The fusion of visual and linguistic intelligence is transforming customer engagement across industries, unlocking revolutionary methods to uncover insights and craft contextualized content. By automating repetitive tasks, employees can now focus on delivering white-glove service that forges lasting bonds with customers. These technologies paint a vivid portrait of human behavior – analyzing filters, walkthroughs, and searches to reveal preferences, needs, and even unspoken wishes. As videos and transcripts reveal nuanced narratives of customer journeys, boundless opportunities emerge to optimize each touchpoint. Ultimately, this synergy of AI capabilities promises more meaningful personal connections, richer insights, and enhanced experiences for businesses and their customers.

References :

Video Analytics Market Size by Straits ResearchGenerative AI Market Size by Precedence ResearchGoogle Cloud Video Intelligence API

Learn more about how Generative AI on Google Cloud can help your organization.

Ensuring compliance with regulatory requirements is crucial for every bank’s business. While financial regulation is a broad area, detecting and preventing market manipulation and abuse is absolutely mission-critical for an investment bank of Deutsche Bank’s size. This is called trade surveillance.

At Deutsche Bank, the Compliance Technology division is responsible for the technical implementation of this control function. To do this, the Compliance Technology team retrieve data from various operational systems in the front office and performs scenario calculations to monitor the trades executed by all of the bank’s business lines. If any suspicious patterns are detected, a compliance officer receives an internal alert to investigate the issue for resolution.

The input data comes from a broad range of systems, but the most relevant are market, trade, and reference data. Historically, provisioning data for compliance technology applications from front-office systems required the team to copy data between, and often even within, many different analytical systems, leading to data quality and lineage issues as well as increased architectural complexity. At the same time, executing trade surveillance scenarios includes processing large volumes of data, which requires a solution that can store and process all the data using distributed compute frameworks like Apache Spark.

A new architectural approach

Google Cloud can help solve the complex issues of processing and sharing data at scale across a large organization with its comprehensive data analytics ecosystem of products and services. BigQuery, Google Cloud’s serverless data warehouse, and Dataproc, a managed service for running Apache Spark workloads, are well positioned to support data-heavy business use cases, such as trade surveillance.

The Compliance Technology team decided to leverage these managed services from Google Cloud in their new architecture for trade surveillance. In the new architecture, the operational front-office systems act as publishers that present their data in BigQuery tables. This includes trade, market and reference data that is now available in BigQuery to various data consumers, including the Trade Surveillance application. As the Compliance Technology team doesn’t need all the data that is published from the front-office systems, they can create multiple views derived from only the input data that includes the required information needed to execute trade surveillance scenarios.

Scenario execution involves running trade surveillance business logic in the form of various different data transformations in BigQuery, Spark in Dataproc, and other applications. This business logic is where suspicious trading patterns, indicating market abuse or market manipulation, can be detected. Suspicious cases are written to output BigQuery tables and then processed through research and investigation workflows, where compliance officers perform investigations, detect potential false positives, or file a Suspicious Activity Report to the regulator if the suspicious case indicates a compliance violation.

Surveillance alerts are also retained and persistently stored to calculate how effective the detection is and improve the rate of how many false positives are actually detected. These calculations are run in Dataproc using Spark and in BigQuery using SQL. They are performed periodically and fed back into the trade surveillance scenario execution to further improve the surveillance mechanisms. Orchestrating the execution of ETL processes to derive data for executing trade surveillance scenarios and effectiveness calibrations is done through Cloud Composer, a managed service for workflow orchestration using Apache Airflow.

Here is a simplified view of what the new architecture looks like:

At Google Cloud we are constantly working on providing capabilities to improve the cost-efficiency of your infrastructure. One of the ways to do that is to stop or suspend your Compute Engine virtual machines (VMs), to avoid compute charges if they are idle. However, if you manage your applications with Managed Instance Groups (MIGs), this capability was not available.

Introducing standby pool in Managed Instance Groups

Now with standby pool in Managed Instance Groups (MIG) you can pause and resume VMs, manually or as part of MIG automation. This is a new way for MIGs to reduce costs when pausing applications, or enable a MIG to respond faster to increased load with pre-initialized VMs.

Why should you try it?

Stopped or suspended VMs cost only a fraction of a price of running VMs, and you can resume them when you need them in the same state as they were before, without losing the VM’s name, IPs, metadata, disks content or even memory. Standby pool in MIG can be used to:

Pause applications and save on costs, rather than running idle VMs and getting charged for compute. VMs in the standby pool keep their state, including software and configurations managed by configuration management tools like Puppet, Chef, Ansible, etc.Stop application and preserve the content of their disks after performing a given calculation, for example at the end of an HPC batch job.Accelerate a MIG’s horizontal scaling cost-effectively, by pre-initializing VMs, pausing them, and resuming them in response to rapid load increase. This is more cost-effective than overprovisioning (running an excessive number of VMs), and depending on the specific application, can also be several times faster than creating a new one and initializing it.

Getting started

You can stop/suspend and start/resume VMs in a MIG individually through the Google Cloud console. In the instance group overview page, select VMs and click the Stop or Suspend button.

You can also stop, suspend, and resume VMs through gcloud and API calls. Read more about manually stopping and suspending VMs.

Finally, you can configure standby pool behavior through the console, gcloud and API. When creating or editing a MIG, go to “advanced configuration” and open the “standby pool” section.

Standby pool has target sizes for the number of suspended and stopped VMs that can be modified. The MIG will automatically create and stop and suspend VMs to meet the requested number of standby VMs.

Standby pool can be configured in two modes:

Manual – you have full control over stopping, suspending and starting VMs, and you can decide which VMs are running, stopped or suspended at any moment.Scale-out-pool – you set the size of the standby pool so that the MIG automatically starts VMs from the standby pool when its size is increased (either by you or by the autoscaler). It then automatically replenishes the standby pool by pre-initializing VMs and putting them in the pool, to be ready for the next load increase. Read more about accelerating MIG scale-out with standby pool.

Further, you can configure how long MIG must wait from the time a VM is initialized to when it is ready to be stopped or suspended by setting Initial delay. This helps ensure that your initialization scripts have the time to complete.

Try standby pool today

Standby pool is in preview across all regions. You can learn more about how to stop/suspend VMs in MIG and how to configure standby pool in the documentation page.

At loveholidays, one of the UK’s fastest-growing online travel agents, we consider ourselves a tech company that sells travel. Our tech stack underpins everything we do. Previously, technology placed a limit on what we could achieve. The two large sales peaks we experience every year — at the end of the year and over the summer — presented a challenge to our former on-premises infrastructure, making it difficult to process bookings efficiently. After prioritizing technology, and moving to Google Cloud, our site now has the scalability and elasticity to meet those peaks effectively, handling a hundredfold increase in demand from our quietest day to our busiest, without any impact on our customers. Technology has set us free in terms of what we can achieve.

Nowhere is that more the case than in our contact center. loveholidays is growing fast. Last year we handled 1,000 contacts a day, now we handle up to 3,000. Scaling at that rate is not only expensive in terms of customer service agents, it is also immensely challenging in terms of operational complexity, and the stakes are high if we are unable to handle that volume of inquiries. Our customers buy protected package holidays, meaning we have a duty of care to our customers. If they are unable to get through to us with their concerns about everything from flight cancellations to natural disasters, it can cause them understandable distress.

Fielding customer inquiries with virtual agents

With our emphasis on choosing the right tech and finding the force multipliers to allow us to leverage small investments for big growth, we turned to Contact Center AI (CCAI), part of Google Cloud’s suite of AI products. Using Dialogflow in CCAI, we created a virtual assistant (VA) that could handle a portion of our inquiries without the need for human agent interaction, which both reduced costs and improved the customer experience. We first trialed our VA during the pandemic, when the main reason for contacting us was to get a refund for a canceled booking, making the inquiries easy to handle and accelerating our responsiveness to customers. As the pandemic eased, and travel returned, customers began calling us for a growing number of reasons, and we expanded the intents in the VA to handle a wider range of calls.

It’s not just what you say, it’s how you say it

Within its first year, our solution was already handling 10% of all customer service traffic, with customers connected to a VA in a fraction of a second, their inquiries generally handled within six branches of the conversational flow, and chats lasting less than 50 seconds. Inspired by this success, we continued to invest in our virtual contact center, with a move to the more advanced Dialogflow CX. This allowed us to build more complexity into our conversational AI, including a type of short-term memory that maintained the context of the customer’s call across different subject areas in the VA. It also gave us visual maps of everything we built to help us monitor and manage the system.

At the same time, we obsess over the language and tone of our VAs, to ensure they provide not only information but reassurance. We do this by repeatedly testing everything our VAs say to our customers. Every time we create a new response to a particular line of inquiry, we observe how well that response converts with our customers. We expect to see around five percent of our customers satisfied with these first versions, and we then take those responses and tweak them to drive up that conversion rate, playing constantly with language and tone until we reach 75% of customers satisfied with the response they receive. And then we move on to honing the next line of response. Thanks to all this development, we now have VAs handling 50% of all our traffic, saving our business £3 million to date.

Reassuring customers with the right agent for each inquiry

With so many calls handled by our VAs, our human agents now have more time to resolve the more sensitive and complex cases. For all other inquiries though, we believe our VAs provide exceptional support. Not only are they able to be trained on, and store, a vast amount of information to quickly and authoritatively answer any query, they also persistently achieve high accuracy. This is backed up by our own data, with our VAs performing consistently highly in our customer satisfaction scores.

Meet Sandy, our voice AI

Following the success of our chat VA, we recently developed a voice agent called Sandy to handle the small portion of inquiries that are made by phone. All of our calls are initially answered by Sandy, which filters them according to the customer’s intent and routes them appropriately. Many are passed to human agents, but we have already managed to fully automate certain call flows, such as those who call because they have entered the wrong email address in their registration and can’t access their booking. Sandy is able to offer all those callers the information they need to resolve their issues, while being an authoritative company voice to ease their concerns.

Shaping our business strategy with profound customer insights

Perhaps the most impactful outcome of our work with Contact Center AI, however, is the data product we have built using CCAI Insights, which gives us a detailed picture of everything our customers want and need. Gone are the days when we would conduct focus groups to understand what our customers really thought. Instead, we capture and categorize data on all customer interactions, chat and voice, that occur every day. We use these insights to shape our business strategy and decide where to invest resources. For example, if we see that a significant portion of customers are experiencing the same problem, such as a lack of information about a particular airline, we can fix it by adding more content to the VA and by building APIs to feed more relevant flight information into our website. Millions of pounds of investment is now driven by the insights we receive from our conversational AI interactions with our customers.

As we continue to develop our VAs in both voice and chat to the point where they can handle almost all inquiries we have noticed a shift in the way we now think about our contact center. We are no longer worrying about whether it will be able to scale to handle the annual spike at Christmas. Instead, we are thinking about how quickly and effectively we will be able to handle each individual customer contact. All of our anxieties about our infrastructure have gone, leaving us to focus exclusively on our customers and their experience of our services. For a company that is growing as quickly as we are, that is an exciting place to be.

Welcome to the first Cloud CISO Perspectives for February 2024. Today I’ll be looking at our latest Threat Horizons report, which provides a forward-thinking view of cloud security with intelligence on emerging threats and actionable recommendations from Google’s security experts.

As with all Cloud CISO Perspectives, the contents of this newsletter are posted to the Google Cloud blog. If you’re reading this on the website and you’d like to receive the email version, you can subscribe here.

–Phil Venables, VP, TI Security & CISO, Google Cloud

Latest Threat Horizons report: Emerging threats, actionable recommendations

By Phil Venables, VP, TI Security & CISO, Google Cloud

Google Cloud’s new Threat Horizons report for the first half of 2024 concludes that we saw threats increase across information technology environments, including on-premise, mobile, operational technology, and the cloud in 2023. However, while some sectors were more plagued by underlying vulnerabilities, security concerns for cloud providers were more often driven by security hygiene and misconfigurations. We expect those trends to continue in 2024, as the report notes, which means that organizations may also have a better sense of the kinds of threats they face (and how to prepare for them) in the coming year.

At Google Cloud, we take credential abuse seriously, so we offer many security capabilities to support customers and protect their environments, including two-factor authentication, strong password policies, identity and access management (IAM) policies, cloud audit logs, and a centralized view of threats and vulnerabilities in Security Command Center. We also now offer a credential leak monitoring service through Mandiant Digital Threat Monitoring.

The report also confirmed that data theft in general and ransomware specifically remain challenges for cloud customers. In 2023, threat actors targeting cloud environments began prioritizing data exfiltration over data encryption and stolen data advertisements. While this could suggest that they are focused on monetary gain, we noted that they “increasingly seek to profit by selling the data (or access to the data) rather than expecting victims to pay the ransom for decryption keys.”

To better secure cloud instances, we recommend deploying cloud-specific backup strategies that include testing configurations and templates of stored assets; using technologies such as WORM (Write Once Read Many) and the Bucket Lock feature on Google Cloud to provide immutable and policy compliant backup storage, and implementing resilient architecture, such as multi-region cloud use and backup mirroring, to reduce risk of data loss or inaccessibility.

Security logs can often capture and make visible to defenders a level of contextual and historical background data that enables them to better observe threat actors at work. Exporting relevant logs to a centralized, well-governed repository can help monitor threat activity on your network and in your systems.

As more industries and organizations move to the cloud, it will become an ever-more appealing target. The full report provides more details on these topics as well as on nation-state threat activity, ransomware, and the positive impact of security partnerships, which you can read here. You can read previous Threat Horizons reports here.

In case you missed it

Here are the latest updates, products, services, and resources from our security teams:

Gen AI governance: 10 tips to level up your AI program: To help organizations navigate AI governance challenges, we’ve outlined 10 best practices to streamline and operationalize AI implementation at scale. Read more.Scaling security with AI, from detection to solution: In alignment with Google’s Secure AI Framework (SAIF), we’ve been using AI itself to automate and streamline routine and manual security tasks, including fixing security bugs. Last year, we wrote about how LLMs can expand vulnerability testing coverage, and we’re excited to share some updates. Read more.U.N. cybercrime treaty could endanger web security: Earlier this month, the United Nations convened member states to continue its years-long negotiations on the U.N. Cybercrime Treaty. Google takes the threat of cybercrime very seriously, and dedicates significant resources to combating it. We urge member states to heed calls from civil society groups to address critical gaps in the treaty and revise the text to protect users and security professionals. Read more.Simplify DORA compliance with Google Cloud’s updated contracts: Google Cloud is committed to supporting our customers’ compliance with EU DORA. Here’s the latest on our DORA support. Read more.A recipe for scaling security: In 2023, Google was able to modernize and improve our code at scale — no small feat given our vast portfolio of services. It’s important for us to share how we did it, what security benefits the changes brought to our users, and importantly, how other interested organizations can adopt the same strategy. Read more.Introducing Policy Analyzer: We are pleased to announce the general availability launch of Policy Analyzer for Org Policy and Custom Org Policy, which can help customers identify which resources are governed by which org policy constraints, empowering customers to understand and strengthen their security posture, and satisfy attestation and auditing requirements. Read more.Synthesized uses gen AI for compliant BigQuery dataset snapshots: Learn how Synthesized uses generative AI to create privacy-preserving snapshots of BigQuery datasets. Read more.

News from Mandiant

Latest research on Ivanti Connect Secure VPN zero-day exploitation: We follow our first blog on two zero days impacting VPN provider Ivanti with additional tactics, techniques, and procedures used by UNC5221 and other threat groups against Ivanti customers. We also detail new malware families and variants to previously-identified malware families being used by UNC5221. Read more.Uncovering USB malware’s hidden depths: Mandiant Managed Defense has been tracking UNC4990, a threat actor who heavily relies on USB devices for initial infection. Since at least 2020, they have primarily targeted users based in Italy and are likely motivated by financial gain. Read more.Introducing credential leak monitoring from Mandiant: Mandiant Digital Threat Monitoring now includes the capability to monitor your credential leaks on the deep and dark web, which will automatically alert you if your employees’ accounts have appeared in our compromised credential data. Read more.Dynamic capa: Exploring executable run-time behavior with CAPE sandbox: We are excited to announce that capa v7.0 can now identify program capabilities from dynamic analysis reports generated from the CAPE sandbox, an expansion of capa’s original static analysis approach. Read more.

Now hear this: Google Cloud Security and Mandiant podcasts

Cloud-ghostbusters: Who you gonna call for cloud forensics: How does cybersecurity forensics happen in the cloud? Google security engineer Jason Solomon sits down with our Cloud Security podcast hosts Anton Chuvakin and Tim Peacock to talk about next steps when there’s something strange in your cloud neighborhood. Listen here.Learn how Google Workspace security is built for modern threats: Workspace makes the claim that, unlike other productivity suites available today, it’s designed for the modern threat landscape. What gives Google the ability to make this claim? Engineering managers Emre Kanlikilicer and Sophia Gu discuss with Anton and Tim what sets Google Workspace apart, from a security perspective. Listen here.Prescriptions for a healthy cybersecurity future: Unfortunately, 2023 was a banner year for healthcare and life sciences cybersecurity incidents. Google Cloud Office of the CISO Director Taylor Lehmann and Security Architect Bill Reid join host Luke McNamara to discuss their takeaways from the last year of threat activity witnessed by enterprises within healthcare and life sciences. Listen here.Is the CTI lifecycle due for an update: Mandiant Intelligence Advisor Renze Jongman joins Luke to discuss his blog post on the CTI Process Hyperloop, and how we can do better at applying threat intelligence to the needs of the security organization and larger enterprise. Listen here.

To have our Cloud CISO Perspectives post delivered twice a month to your inbox, sign up for our newsletter. We’ll be back in two weeks with more security-related updates from Google Cloud.