GCP – Enhancing multiplayer games with gen AI and cloud gaming: a technical deep dive

Generative AI is poised to revolutionize the gaming landscape. It will offer — and in many cases, already is offering — personalized NPCs, dynamic narratives, and tailored environments, thereby reshaping player experiences. It’s a concept increasingly known as living games.

The recent unveiling of Genie 2: a large scale foundation world model created by Google DeepMind exemplifies this potential. Living cloud gaming, underpinned by Kubernetes, Agones, and Open Match, and integrated with generative AI, can extend these advancements to a vast audience without sacrificing performance or user experience.

In this blog post, we’ll take you on a journey of integrating cutting-edge generative AI models — like Google’s Gemini or open models such as Gemma — into scalable multiplayer game servers powered by Google Kubernetes Engine (GKE) and Agones. This blog will showcase how a sample game can be enhanced with generative AI, enabling developers to enrich their games with AI-driven features such as interactive user guidance, real-time chat capabilities, and personalized summaries of user progress, all while maintaining scalability and reliability.

Cloud gaming with Agones: the foundation for scalable multiplayer experiences

Agones is an open-source, Kubernetes-based game-server-hosting solution that enables developers to efficiently manage large numbers of dedicated game servers. While Agones works on any Kubernetes deployment, running it on GKE provides Google Cloud customers with official support.

Agones also integrates with Open Match, an open-source matchmaking framework that simplifies the creation of multiplayer matches based on player criteria such as skill level or location. 

This combination of Agones and Open Match provides a powerful foundation for scalable cloud games.

Generative AI’s Potential in Cloud Gaming

By utilizing generative AI in real time, living games can go beyond traditional live services, creating a dynamic and engaging player experience without constant developer input. Living games can foster a deep connection between players and the game world, ensuring that each playthrough is unique, through features like dynamic storyline adaptation, responsive NPC interactions, and personalized in-game content.

Building a language learning game with gen AI: A technical walkthrough

To illustrate a simple example of Agones and generative AI integration, we created a language learning game called Lexitrail. Lexitrail helps players memorize Chinese characters in an interactive way while also competing against other players in multiplayer mode.

Lexitrail utilizes a React frontend, a Python Flask backend, and a MySQL database. All of them are deployed in-cluster on Google Kubernetes Engine (GKE). A key component is the game session management layer, deployed as Agones game servers for seamless game session handling.

Within Lexitrail, we leverage the power of Gemini and Imagen models. These models generate subtle, ambiguous hint images that aid players in memorizing and recalling words. To further personalize the experience, we provide additional user-specific context alongside the LLM prompts.

High-Level Architecture of Lexitrail

The diagram below illustrates the fundamental architecture and components that underpin the Lexitrail system.

This architecture is designed to provide key features that enhance the gaming experience:

• Seamless game sessions: Utilizing Agones for game session management ensures uninterrupted gameplay with quick state transitions in both single-player and multiplayer modes.
• Integration of generative AI: The generative AI integration layer’s adaptability enables developers to create endpoints for both closed models (e.g. Gemini, Imagen) and open models (e.g., Llama, Stable Diffusion, Gemma). These endpoints can then be integrated with other components in the game application, allowing for the creation of detailed and suggestive hint images specifically tailored to the individual game’s requirements
• Skill-based matchmaking: In multiplayer environments, Open Match creates balanced and engaging matches by pairing players based on their skill levels.
• Scalability and reliability: The foundation of this architecture is GKE, providing the Kubernetes infrastructure necessary for managing and autoscaling game servers. Running Agones and Open Match on GKE guarantees that the language learning game can effortlessly support a global player base, from small study groups to massive tournaments.

Here’s a breakdown of the Lexitrail game application’s architecture, designed for a smooth and scalable multiplayer experience:

• Lexitrail frontend: This is the initial interface for players. It manages authentication through Google login. The user interface component and the business logic governing its interactions are both managed within the frontend. Various game modes are provided. Players can practice solo or compete against others in multiplayer matches. Additionally, players can select wordsets based on different HSK (Hanyu Shuiping Kaoshi) levels for each game mode. When a player opts for multiplayer mode, the game’s frontend triggers the matchmaking process. This process involves sending matchmaking requests to the Open Match component. 
• Open Match: Lexitrail employs Open Match which is deployed as a GKE service for its matchmaking system. The current setup is straightforward: players who select the same wordset are paired together. However, Open Match’s flexibility allows developers to implement more intricate matchmaking algorithms tailored to their specific game requirements. Once a match is determined, Open Match collaborates with the game server orchestration system to designate a dedicated game server for that match. Then the matchmaker passes back to the game clients (through the frontend) the address and port each client will need to connect directly to, since the game state is an in-memory simulation, and then, they play Lexitrail!
• Game server: In Lexitrail, the game server is responsible for managing game sessions for pairs of players. It oversees gameplay logic, tracks intermediate word recall results, and maintains the final scoreboard. Additionally, the game server enhances performance by acting as a buffer layer. It batches time-consuming tasks, such as database updates and asynchronously flushes them to the backend, ensuring that these operations don’t negatively impact the responsiveness of the frontend.
• Agones: Lexitrail employs Agones for game server orchestration, leveraging its allocator service to facilitate matchmaker’s requests to get the dedicated game server for each match. Agones manages these game servers to ensure availability so the player experiences aren’t interrupted while players are playing on them. Additionally, Lexitrail utilizes Agones’ fleet autoscaler to dynamically adjust the number of game servers based on demand.
• Game backend: Lexitrail’s game backend, deployed as a GKE service, manages the API interface and business logic for database interactions. It handles requests from both the frontend and game servers, retrieving and updating player data like wordsets, words, and recall history. Additionally, the backend integrates with the generative AI endpoints to generate hint texts and images that aid players in memorizing and recalling words.
• Database: Lexitrail’s backend database is powered by MySQL, deployed within the GKE environment. This database houses all essential game data which includes user profiles, word sets, individual word entries, and player recall history. The database schema can be found here.
• Generative AI integration: In the game’s user interface, players are presented with words to memorize during gameplay, accompanied by visual hints to aid in recall. These image hints are generated by integrating with generative AI endpoints, which are deployed within the GKE environment. The game backend queries the image generation model endpoint to create the hint images for each word. The flexibility of kubernetes based architecture allows developers to easily select and utilize different underlying LLMs for image generation. 

Take the Next Step

The potential for gen AI in gaming is enormous, and the tools are now more accessible than ever. We invite you to explore the repo as a starting point.

• Explore gen AI integration: Start integrating gen AI models into your own applications using Agones and GKE.
• Leverage Vertex AI and GKE models: Experiment and compare integration between Vertex AI or open models on GKE to see how AI can transform your gaming experience.
• Join the community: Engage with the developer community for Agones, Open Match, and broader GKE AI projects — share your insights, learn from others, and help shape the future of AI-powered gaming.

Stay at the forefront of gaming technology by embracing scalable AI-driven features that deliver new levels of personalization and engagement to players, bringing living games to reality. 

Kubernetes and GKE with Agones and Open Match provide a scalable, production-proven enterprise-grade solution for cloud games that leverage state-of-the-art components. The journey of scalable, AI-powered gaming is just beginning — and we can’t wait to see what you build!