Cloud

One of the most compelling aspects of cloud computing is being able to automatically scale resources up, but almost as importantly, to scale them back down to manage costs and performance. This is standard practice with virtual machines, for instance Compute Engine Managed Instance Groups, but because of their inherent complexity, less so with stateful services such as databases.

Last year we released Memorystore for Redis Cluster with the ability to manually trigger scale out and down. Today, to meet the incredibly elastic nature of modern Memorystore workloads, we’re excited to announce the open-source Memorystore Cluster Autoscaler available on GitHub, which builds on our open-source Spanner Autoscaler, which we released in 2020.

Understanding cluster scaling

Memorystore for Redis Cluster capacity is determined by the number of shards in your cluster, which can be increased/decreased without downtime, and your cluster’s shard size, which maps on to the underlying node type. At this time, the node type of the cluster is immutable. To scale capacity in or out, you modify the number of shards in your cluster. To automate this process, you can deploy the Memorystore Cluster Autoscaler to monitor your cluster metrics, and rightsize your cluster based on that information. The Autoscaler performs the necessary resource adjustments using rulesets that evaluate memory and CPU utilization, without impacting cluster availability.

The following chart shows the Autoscaler in action, with a Memorystore for Redis Cluster instance automatically scaling out as memory utilization increases. The green line represents data being written to the cluster at the rate of one gigabyte every five minutes. The blue line represents the number of shards in the cluster. You can see that the cluster scales out, with the number of shards increasing in proportion to the memory utilization, then plateaus when the writes stop, and finally scales back in when the keys are flushed at the end of the test.

Experience and deployment

To use the Autoscaler, deploy it to one of your Google Cloud projects. The Autoscaler is very flexible and there are multiple options for its deployment, so the repository contains multiple example Terraform deployment configurations, as well as documentation that describes the various deployment models.

Once you’ve deployed the Autoscaler, configure it according to the scaling requirements of the Memorystore instances being managed, to suit your workloads’ characteristics. You do this by setting Autoscaler configuration parameters for each of the Memorystore instances. Once configured, the Autoscaler autonomously manages and scales the Memorystore instances. You can read more about these parameters later in this post, and in the Autoscaler documentation.

Autoscaler architecture

The Autoscaler consists of two main components, the Poller and the Scaler. You can deploy these to either Cloud Run functions or Google Kubernetes Engine (GKE) via Terraform, and configure them so that the Autoscaler runs according to a user-defined schedule. The Poller queries the Memorystore metrics in Cloud Monitoring at a pre-defined interval to determine utilization, and passes them to the Scaler. The Scaler then compares the metrics against the recommended thresholds specified in the rule set, and determines if the instance should be scaled in or out, and if so, by how many shards. You can modify the sample configuration to determine minimum and maximum cluster sizes and any other thresholds suitable for your environment.

Throughout the flow, the Autoscaler writes a step-by-step summary of its recommendations and actions to Cloud Logging for tracking and auditing, as well as metrics to Cloud Monitoring to provide insight into its actions.

Scaling rubrics

Memorystore performance is most commonly limited by in-memory storage and by CPU. The Autoscaler is configured by default to take both of these factors into consideration when scaling, by utilizing the CPU_AND_MEMORY profile. This is a good place to start your deployment, and can be replaced with a custom configuration, if required, to best suit your needs.

Defaults

* Scale-in will be blocked if there are ongoing key evictions, which occur when the keyspace is full and keys are removed from the cache to make room. Scale in is enabled by default, but can be configured using a custom scaling profile. Refer to the Scaling Profiles section of the documentation for more information on how to do this.

Scaling scenarios and methods

Let’s take a look at some typical scenarios and their specific utilization patterns, and the Autoscaler configurations best suited to each of them. You can read more about the options described in the following section in the configuration documentation.

Standard workloads

With many applications backed by Memorystore, users interact with the application at certain times of day more than others, in a regular pattern — think a banking application where users check their accounts in the morning, make transactions during the afternoon and early evening, but don’t use the application much at night. 

We refer to this fairly typical scenario as a “standard workload” whose time series shows:

• Large utilization increase or decrease at certain points of the day

• Small spikes over and under the threshold

A recommended base configuration for these types of workflow should include:

• The LINEAR scalingMethod to cover large scale events

• A small value for scaleOutCoolingMinutes — between 5 and 10 minutes — to minimize Autoscaler’s reaction time.

Plateau workloads

Another common scenario is applications with more consistent utilization during the day such as global apps, games, or chat applications. User interactions with these applications are more consistent, so the jumps in utilization are less pronounced than for a standard workload.

These scenarios create a “plateau workload” whose time series shows:

• A pattern composed of various plateaus during the day

• Some larger spikes within the same plateau

A recommended base configuration for these types of workflow should include:

• The STEPWISE scalingMethod, with a stepSize sufficient to cover the largest utilization jump using only a few steps during a normal day, OR

• The LINEAR scalingMethod, if there is likely to be a considerable increase or reduction in utilization at certain times, for example when breaking news is shared. Use this method together with a scaleInLimit to avoid reducing the capacity of your instance too quickly

Batch workloads

Customers often need increased capacity for their Memorystore clusters to handle batch processes or a sales event, where the timing is usually known in advance. These scenarios comprise a “batch workload” with the following properties:

• A scheduled, well-known peak that requires additional compute capacity

• A drop in utilization when the process or event is over

A recommended base configuration for these types of workloads should include two separate scheduled jobs:

• One for the batch process or event, that includes an object in the configuration that uses the DIRECT scalingMethod, and a minSize value of the peak number of shards/nodes to cover the process or event

• One for regular operations, that includes configuration with the same projectId and instanceId, but using the LINEAR or STEPWISE method. This job will take care of decreasing the capacity when the process or event is over

Be sure to choose an appropriate scaling schedule so that the two configurations don’t conflict. For both Cloud Run functions and GKE deployments, make sure the batch operation starts before the Autoscaler starts to scale the instance back in again. You can use the scaleInLimit parameter to slow the scale-in operation down if needed.

Spiky workloads

Depending on load, it can take around several minutes for Memorystore to update the cluster topology and fully utilize new capacity. Therefore, if your traffic is characterized by very spiky traffic or sudden-onset load patterns, the Autoscaler might not be able to provision capacity quickly enough to avoid latency, or efficiently enough to yield cost savings.

For these spiky workloads, a base configuration should:

• Set a minSize that slightly over-provisions the usual instance workload

• Use the LINEAR scalingMethod, in combination with a scaleInLimit to avoid further latency when the spike is over

• Choose scaling thresholds large enough to smooth out some smaller spikes, while still being reactive to large ones

Advanced usage

As described above, the Autoscaler is preconfigured with scaling rules designed to optimize cluster size based on CPU and memory utilization. However, depending on your workload(s), you may find that you need to modify these rules to suit your utilization, performance and/or budget goals.

There are several ways to customize the rule sets that are used for scaling, in increasing order of effort required:

1. Choose to scale on only memory or only CPU metrics. This can help if you find your clusters flapping, i.e., alternating rapidly between sizes. You can do this by specifying a scalingProfile of either CPU or MEMORY to override the default CPU_AND_MEMORY in the Autoscaler configuration.

2. Use your own custom scaling rules by specifying a scalingProfile of CUSTOM, and supplying a custom rule set in the Autoscaler configuration as shown in the example here.

3. Create your own custom rule sets and make them available for everyone in your organization to use as part of a scaling profile. You can do this by customizing one of the existing scaling profiles to suit your needs. We recommend starting by looking at the existing scaling rules and profiles, and creating your own customizations.

Next steps

The OSS Autoscaler comes with a Terraform configuration to get you started, which can be integrated into your codebase for production deployments. We recommend starting with non-production environments, and progressing through to production when you are confident with the behavior of the Autoscaler alongside your application(s). Some more tips for production deployments are here in the documentation.

If there are additional features you would like to see in the Autoscaler — or would like to contribute to it yourself — please don’t hesitate to raise an issue via the GitHub issues page. We’re looking forward to hearing from you.

Today, we are thrilled to announce the public beta launch of Gen AI Toolbox for Databases in partnership with LangChain, the leading orchestration framework for developers building large language model (LLM) applications.

Gen AI Toolbox for Databases (Toolbox) is an open-source server that empowers application developers to connect production-grade, agent-based generative AI (gen AI) applications to databases. It streamlines the creation, deployment, and management of sophisticated gen AI tools capable of querying databases with secure access, robust observability, scalability, and comprehensive manageability. It also provides connectivity to popular open-source databases such as PostgreSQL, MySQL, as well as Google’s industry-leading Cloud Databases like AlloyDB, Spanner, and Cloud SQL for SQL Server. We are open to contributions from other databases outside of Google Cloud.

In this post, we’ll explore how Gen AI Toolbox for Databases works, and how to get started.

Challenges in gen AI tool management

Building AI agents requires using different tools, frameworks, and connecting to various data sources. This process creates several challenges for developers, particularly when these tools need to query databases. These include –

• Scaling tool management: Current approaches to tool integration often require extensive, repetitive code and modifications across multiple locations for each tool. This complexity hinders consistency, especially when tools are shared across multiple agents or services. A more streamlined framework integration is needed to simplify tool management and ensure consistency across agents and applications.

• Complex database connections: Databases require configuration, connection pooling, and caching for optimal performance at scale.

• Security vulnerabilities: Ensuring secure access from gen AI models to sensitive data requires complex integration with auth services, databases and the application, which can be error-prone and introduce security risks.

• Inflexible tool updates: Adding new tools or updating existing ones often requires a complete rebuild and redeployment of the application, potentially leading to downtime.

• Limited workflow observability: Current solutions lack built-in support for comprehensive monitoring and troubleshooting, making it difficult to gain insights into gen AI workflows with databases.

Components

Gen AI Toolbox for Databases improves how gen AI tools interact with data, addressing common challenges in gen AI tool management. By acting as an intermediary between the application’s orchestration layer and data sources/databases, it enables faster development and more secure data access, improving the production-quality of tools.

Toolbox comprises two components: a server specifying the tools for application use, and a client interacting with this server to load these tools onto orchestration frameworks. This centralizes tool deployment and updates, incorporating built-in production best practices to enhance performance, security, and simplify deployments.

Benefits

Toolbox offers various features that provide better managebility, security and observability for AI Agents. Some of the benefits for application developers are as follows – 

1. Simplified development – Reduced boilerplate code and consolidated integration simplifies tool development and sharing across other agents.
2. Built-in performance and scale – Built-in connection pooling and optimized connectors for popular databases to handle connection management efficiency.
3. Zero downtime deployment – A config-driven approach enables seamless deployment of new tools and updates without any service interruption and supports incremental rollouts.
4. Enhanced security – Using Oauth2 and ODIC, built-in support for common auth providers enables control over Agents’ access to tools and data.
5. End-to-end observability – Toolbox integrates with OpenTelemetry, providing day-one insights via logging, metrics, and tracing, offering end-to-end observability for better operations.

Compatibility with LangChain

LangChain is the most popular developer framework for building LLM applications, and we’re excited to announce Toolbox compatibility with the LangChain ecosystem from day one. Together with Toolbox, LangGraph can leverage LLMs like Gemini on Vertex AI to build powerful agentic workflows.

LangGraph extends LangChain’s capabilities by providing a framework for building stateful, multi-actor applications with LLMs. Its support for cycles, state management, and coordination enables the development of complex and dynamic AI agents. All of these capabilities integrate seamlessly with Toolbox.

Tool calling is essential for building agents. Agents need to call tools in a controlled and specified way, run the tool reliably, and then pass the correct context back to the LLM. LangGraph provides a low-level agent framework for managing how tools are called and how their responses are integrated, ensuring precision and control. Toolbox then handles the execution itself, seamlessly running the tool and returning results. Together, they create a powerful solution for tool calling in agent workflows.

“The integration of Gen AI Toolbox for Databases with the LangChain ecosystem is a boon for all developers” says Harrison Chase, CEO of LangChain. “In particular, the tight integration between Toolbox and LangGraph will allow developers to build more reliable agents than ever before.”

Get started with Gen AI Toolbox for Databases

Gen AI Toolbox for Databases simplifies gen AI tool development and deployment by automating the entire lifecycle. Here are some resources to get you started:

• Github repo

• Documentation

• Quickstart – How to get started running a LangGraph agent with Toolbox using Gemini on Vertex AI.

Last year, we offered our first ever “Google Launchpad for Women” series to empower women within our customer ecosystem to grow their cloud and AI skills. The response from our customers has been tremendous: more than 11,000 women across a breadth of roles – sales, leadership, marketing, finance, and more have completed previous editions of the program. As a result, they are building critical skills that help them put AI to work in their jobs, grow their careers, and help transform their businesses.

This year, in honor of International Women’s Day, we are opening “Google Launchpad for Women,” to thousands of more customer participants, providing them with no-cost training, exam prep, and access to Google experts. Registration is now open to Google Cloud customers in the Americas, EMEA, and Japan, with the three-week program beginning on March 4th in Japan and March 6th in the Americas and EMEA. Program benefits include:

• Expert-led training: Two days of in-depth, instructor-led training covering key cloud concepts and best practices.

• Industry insights: Engage with Google Cloud experts through panel discussions on topics such as Generative AI.

• Exam preparation: Dedicated sessions to prepare for the Cloud Digital Leader certification exam.

• Complimentary exam voucher: Participants will receive a voucher for the $99 exam fee.

Why these trainings are critical

Harnessing the power of cloud computing and AI is essential for all job roles, not just IT. As more businesses adopt AI, people across business roles utilize this technology every day and often make purchasing decisions about new AI platforms and tools. However, a talent gap remains, and is particularly pronounced for women, who represent about 14% of the global cloud workforce according to recent data from the World Economic Forum.

We aim to help our customers reduce this gap, ensure they have access to the skilled experts they need to advance their digital and AI transformations, and give more people opportunities to grow their careers and lead these transformations. Ultimately, those who complete the Google Launchpad for Women program will be well-equipped to achieve the Cloud Digital Leader certification, putting them at the forefront of the cloud and AI era.

Google Launchpad for Women is open to all Google Cloud customers, regardless of prior technical experience or role. We welcome women from all professional backgrounds who are eager to develop their cloud skills and advance their careers. While this initiative is specifically focused on women, we invite everyone to participate.

Sign up today

Visit the links below to learn more about each regional session and contact your sales rep to sign up today.

• [Americas Session] Google Launchpad for Women

• [EMEA Session] Google Launchpad for Women

• [Japan Session] Google Launchpad for Women

Mobile devices have become the go-to for daily tasks like online banking, healthcare management, and personal photo storage, making them prime targets for malicious actors seeking to exploit valuable information. Bad actors often turn to publishing and distributing malware via apps as a lucrative channel for generating illegal and/or unethical profits. 

Android takes a multi-layered approach to combating malware to help keep users safe (more later in the post), but while we continuously strengthen our defenses against malware, threat actors are persistently updating their malware to evade detection. Malware developers used to complete their entire malicious aggression using the common Android app development toolkits in Java, which is easier to detect by reversing the Java bytecode. In recent years, malware developers are increasing the use of native code to obfuscate some of the critical malware behaviors and putting their hopes on obscuration in compiled and symbol-stripped Executable and Linkable Format (ELF) files, which can be more difficult and time-consuming to reveal their true intentions.

To combat these new challenges, Android Security and Privacy Team is partnering with Mandiant FLARE to extend the open-source binary analysis tool capa to analyze native ARM ELF files targeting Android. Together, we improved existing and developed new capa rules to detect capabilities observed in Android malware, used the capa rule matches to highlight the highly suspicious code in native files, and prompted Gemini with the highlighted code behaviors for summarization to enhance our review processes for faster decisions.

In this blog post, we will describe how we leverage capa behavior-detection capabilities and state-of-art Gemini summarization by:

• Showcasing a malware sample that used various anti-analysis tricks to evade detections

• Explaining how our existing and new capa rules identify and highlighted those behaviors

• Presenting how Gemini summarizes the highlighted code for security reviews

An Illegal Gambling App Under a Music App Façade

Google Play Store ensures all published apps conform to local laws and regulations. This includes gambling apps, which are prohibited or require licenses in some areas. Developing and distributing illegal gambling apps in such areas can generate significant illicit profits, which sometimes is associated with organized crimes. To bypass Google Play Store’s security-screening procedures, some gambling apps disguise themselves with harmless façades like music or casual games. These apps only reveal their gambling portals in certain geographic markets using various anti-analysis tricks. Unfortunately, dynamic analysis, such as emulation and sandbox detonation, relies on specific device configurations, and threat actors keep trying different combinations of settings to evade our detections. It’s an ongoing game of cat and mouse!

In response, the Android Security and Privacy Team has evolved static analysis techniques, such as those that evaluate the behavior of a complete program and all its conditional logic. So, let’s describe an app that violated Google Play Store rules and show how we can better detect and block other apps like it.

We received reports of a music app opening gambling websites for users in certain geographical areas. It used an interesting trick of hiding key behaviors in a native ELF file that has most symbols (except the exported ones) stripped and is loaded at runtime to evade detection.

When we decompiled the app into Java source code, using a tool like JEB Decompiler, we found that the app has a song-playing functionality as shown in “MainActivity” of Figure 1. This looks like benign behavior and is fully within the limits of Google Play Store policies.

However, there was a small region of initialization code that loads an ELF file as soon as the app is initialized when calling the onCreate function, as shown in com.x.y.z class of Figure 1. To fully understand the behavior of the entire app, we also had to reverse engineer the ELF file, which requires a completely different toolset.

Using a tool like Ghidra, we decompiled the ARM64 ELF file into C source code and found that this app estimates the user’s geographic location using timezone information (“Code Section 1” in Figure 1). The code implements a loop that compares the user’s timezone with a list of target regions (“Data Section” in Figure 1).

If the user’s location matches a value in the list (“Data Section” in Figure 1), this malware:

1. Downloads an encrypted DEX file from a remote server (“Code Section 2” in Figure 1)

2. Decrypts the downloaded DEX file (“Code Section 3” in Figure 1)

3. Loads the decrypted DEX file into memory (“Code Section 4” in Figure 1)

The loaded DEX file uses further server-side cloaking techniques and finally loads a gambling website (Figure 3) to the app users. Compared to the app icon in Figure 2, it is an obvious mismatch of the app’s advertised functionality.

While there are many detection technologies, such as YARA, available for identifying malware distributed in ELF files, they are less resilient to app updates or variations introduced by threat actors. Fortunately, the Android Security and Privacy Team has developed new techniques for detecting malicious Android apps by inspecting their native ELF components. For example, in the gambling app in Figure 3, there are many API calls dynamically resolved via the Java Native Interface (JNI) that interact with the Android runtime. Our detection systems recognized these cross-runtime interactions and reason about their intent. We’ve enumerated behaviors commonly seen in Android malware, such as making ptrace API calls, extracting device information, downloading code from remote servers to local storage, and making various cryptographic operations via JNI, turning them into capa detections we can use to identify and block Google Play Store threats.

Let’s now talk a little more about how this works.

Android capa Rules

capa is a tool that detects capabilities in executable files. You run it against a compiled program, and it tells you what it thinks the program can do. For example, capa might suggest that a file is a backdoor, is capable of installing services, or relies on HTTP to communicate.

Mandiant FLARE extended capa to support BinExport2, an architecture agnostic representation of disassembled programs. This enables capa to match capabilities for additional architectures and file formats, such as those supported by Ghidra and its BinExport2 plugin, with an initial focus on ARM64 ELF files. The Android Security and Privacy Team then created new capa rules focused specifically on detecting capabilities observed in ARM64 ELF files used by various Android malware samples. These proprietary rules alongside capa’s open-source rules are used to detect malware capabilities as part of internal Android malware analysis pipelines.

Referring back to the gambling app in Figure 3, the following Google proprietary rules and open-source capa rules matched the malicious functions performing cloaking techniques for further inspection.

Proprietary rules:

• Make ptrace API calls

• Extract device configuration information via JNI on Android

• Extract timezone via JNI on Android

• Encode or decode data using Base64 via JNI on Android

• Encrypt or decrypt data using Cipher API via JNI on Android

Open-source capa rules:

• Create or open file

• Write file on Linux

• Read file on Linux

• Check file permission on Linux

• Create thread

• Reference Base64 string

Instead of browsing hundreds of thousands lines of obfuscated code, our analysts were able to quickly identify the evidence of the app’s wrong-doings using the function addresses matching those rules and enforced on the app.

Gemini Summaries of capa Rule Matches

Safeguarding the Android ecosystem, our Android malware analysis pipelines scan millions of ELF files in-depth every day, each one containing thousands to millions of lines in their decompiled codes. On top of the fast-evolving Gemini capabilities in malware analysis, capa rules are able to select the most interesting code for Gemini summarization, with sharpened focus on a much smaller set of the most suspicious functions.

We asked Gemini to summarize the functions matched on capa rules from the earlier gambling app with the following prompt:

Gemini responded with the following suggestions:

As seen in the Gemini output, the Android ELF behaviors are explained clearly on the functions matched on capa rules.

In this particular example, Gemini helped to:

• Accentuate the function call sequences to perform dynamic code loading, where our analysts can easily inspect the key function calls getCacheFilePath and getDexClassLoader

• Identify the timezone extraction with the additional URL parameter hint, where our analysts may try to probe the malicious payload quickly and accurately

• Describe more potential suspicious behaviors (e.g. getDexClassLoader JNI call, URL obfuscation) for further rule-writing ideas

capa rules in Android together with Gemini summarization shows great potential for further malware detection with more advanced techniques. Our analysts are closely monitoring the malware trends and techniques in the market and writing up-to-date capa rules to catch the bad actors in the wild.

Android’s Multi-Layered Security Approach

Android’s ever-evolving, multi-layered security approach includes integrating advanced features and working with developers and device implementers to keep the Android platform and ecosystem safe. This includes, but is not limited to:

• Advanced built-in protections: Google Play Protect automatically scans every app on Android devices with Google Play Services, no matter the download source. This built-in protection, enabled by default, provides crucial security against malware and unwanted software. Google Play Protect scans more than 200 billion apps daily and performs real-time scanning at the code-level on novel apps to combat emerging and hidden threats, like polymorphic malware. In 2024, Google Play Protect’s real-time scanning identified more than 13 million new malicious apps from outside Google Play. 

• Google Play and developer protections from malware: To create a trusted experience for everyone on Google Play, we use our SAFE principles as a guide, incorporating multi-layered protections that are always evolving to help keep Google Play safe. These protections start with the developers themselves, who play a crucial role in building secure apps. We provide developers with best-in-class tools, best practices, and on-demand training resources for building safe, high-quality apps. Every app undergoes rigorous review and testing, with only approved apps allowed to appear in the Play Store. Before a user downloads an app from Play, users can explore its user reviews, ratings, and Data safety section on Google Play to help them make an informed decision. 

• Engagement with the security research community: Google works closely with the security community on multiple levels, including the App Defense Alliance, to advance app safety standards. Android also collaborates with Google Threat Intelligence Group (GTIG) to address emerging threats and safeguard Android users worldwide.

Equipped with the fast-evolving Gemini, our analysts are able to spend less time on those sophisticated samples, minimising the exposure for malicious apps and ensuring the safety of Android ecosystems.

Acknowledgement

Special thanks to Willi Ballenthin, Yannis Gasparis, Mike Hunhoff, and Moritz Raabe for their support.