Cloud

How do you like to learn a new skill? Read books? Take a course? Try to build something? We are a small group of Customer Engineers who, with no formal background in machine learning, who decided to learn about reinforcement learning by entering Battlesnake, an online survival game where our trained agent would cut its teeth against competitors around the world. 

Our project was fueled by Vertex AI, Google Cloud’s machine learning platform, and we share our process and lessons learned here in the hope it may inspire you to also learn by doing.

Let’s consider a different approach utilizing the construct of a game to evaluate new technology and learn new skills.  

Enter the arena

Battlesnake isn’t your indestructible Nokia candy bar CDMA phone snake game. This isn’t even an updated Google Snake spin off (but do try and get the secret rainbow snake), this is something very different and much more useful.

On the surface, Battlesnake seems like a simple game with a small number of basic rules:

Don’t run into walls or other snakes

Don’t starve

Don’t get eaten by another snake

Once you break through the basic premise, you’ll soon realize it is a lot more complicated than that.

There are many ways to build and place your own battlesnake into a competition. Depending on your team’s experience level you may want to try out one of the starter projects that Battlesnake makes available.  Alternatively, you may want to start wading into the deeper end of the competitive pool and enhance your snake with health-based heuristics models or cannonball into the pool using a reinforcement learning approach.

The approach we took to our first competition was to hedge our bets a little – get something into competition quickly and gather some data to iterate on, then explore improvements on the initial snake performance through a series of ML model tweaks; ultimately building a reinforcement learning model that we were sure was going to win (in the most virtuous and collaborative sporting way of course).  More on results later but here is walkthrough of how our architecture and development progressed:

Introduction to reinforcement learning

Reinforcement learning (often referred to as RL) has had a long history as a way to build AI models. From games like chess, Go and Starcraft II to more industry specific problems like manufacturing and supply chain optimization, reinforcement learning is being used to build best in class AI to tackle increasingly difficult challenges. 

For those unfamiliar with RL, here is a quick primer:

Traditionally, machine learning models learn to make predictions based on massive amounts of labeled example data. In RL, agents learn through experimentation..
Each iteration is scored based on a reward function. As an example for Battlesnake, a basic set of rewards might be a 1 for winning and a -1 for losing. 
The rewards are fed into the model so that it “learns” which moves earn the highest reward in any given scenario. Similar to humans learning to not touch a hot stove, the model learns that running a snake head first into a wall will produce a negative reward and the model will remember not to do that (most of the time). 
For complex systems this reward structure might consist of dozens of different inputs that help to shape the reward based on the current state of the overall system.

Our team did not have a classically trained machine learning expert but we did have enough expertise to take some concepts that we learned from others who had attempted this approach and apply them using Google Cloud’s Vertex AI platform.

How wecharmedtrained our snake

One of the key starting areas for building a RL model is to set up an environment that knows how to play the game. OpenAI’s gym toolkit provides an easy way for developers to get started building RL models with a simple interface and many examples to start training  your model quickly. This allows you to focus purely on the parts of the model that matter, like….

For our Battlesnake competition, we had one week to build, train and test a snake before throwing it into the arena against our peers in a winner takes all tournament. Though traditional ML loops can take weeks or months to build, with the power and simplicity of Vertex AI, we made an end-to-end ML pipeline in a few hours. This velocity freed up several days to run training experiments and tune our model. 

Training on Vertex AI, in this case, started with a custom training job. Vertex AI allows you to specify your own training container (in our case we used Pytorch) to run any arbitrary training tasks on the Vertex AI managed infrastructure. Initial experimentation started in a basic Jupyter notebook hosted on Vertex AI but we quickly transitioned to a custom Docker container that was produced by Cloud Build on a push to our main source repository. We attempted to source our container image from multiple prebuilt images, but ultimately found that we were spending too much time working through conflicting dependencies which was slowing down our progress so we switched to a cleaner base image containing just the NVIDIA CUDA drivers.

As we started to scale our training we converted our notebook into an ML pipeline using Vertex AI’s managed pipeline service. The workflow took advantage of the built in hyperparameter tuning service to automatically tune our model to perform as well as possible with minimal supervision from our team. Each training job ran for 2-4 hours before going into the evaluation phase. During evaluation we selected previous versions of the model to run simulated games against to see if the newly trained model performed better than previous versions. The winner of the evaluation battles was promoted to the top model and the new baseline for the next iteration of training. Our pipeline was triggered on demand to do additional training. This was usually after we loaded the new model into the snake in the global arena and observed its behavior. If we were taking this fully to production we would have used the data we captured about game win rates to intelligently trigger additional training if the model was starting to underperform.

Deployment architecture

We deployed our top model to production using Vertex AI’s model serving infrastructure. The data sent from Battlesnake is a JSON document of all the current positions of the snakes, the food and any hazards that are on the board.

Responses from our server were required to be completed in 500ms. Any snake that “timed out” responding to requests from the game would continue moving in the same direction that it was already traveling. Given the time constraint we took a two pronged approach to control our snake. Requests from Battlesnake first came to a web service deployed on Google App Engine. The App Engine server forwarded the request to the model server with a hard timeout of 300ms. Responses from the model were validated in the App Engine server as a last fail safe, in case the model responded with a move that was clearly invalid (move outside of the board, etc.). If the model service timed out, the App Engine server fell back on its own logic to produce a reasonable move.

Data analysis

Our snake played thousands of games within the Battlesnake ecosystem both in the public arena and our private tournament. Each game consisted of tens to hundreds of moves and produced a finite result. As part of our experimentation we captured all the messages sent from the Battlesnake platform and fed them via Pub/Sub to BigQuery to be used later for statistical analysis of our performance. As mentioned in the previous sections, the data received from Pub/Sub was not in the standard tabular format but in an array of JSON objects. To perform analysis we first needed to convert the data into a standard table format. We leveraged the UNNEST function in BigQuery to flatten the record into the table. The data points in the messages from the battles consisted of the health, length, food location, head and body coordinates, turns and latency for all the snakes on the board including board size etc. For exploratory analysis of our ML model we attempted to answer questions like which snake did we battle the most and how did we perform in each of those battles? Is there a particular snake that keeps beating us? What are the frequent food locations? After gathering these insights, we can further refine how we tune our model parameters to what works best for our snake.

Conclusion

Our snake performed very well in open competition in the public arena but had a string of bad luck in actual tournament play. During the group stages we competed against two of the snakes that made it into the finals. The primary snake in our group was a hungry monster that quickly gobbled up food and boxed our snake out very effectively. This led to our untimely death by wall collision and a fast exit from tournament play. We accomplished a reasonable amount of success in the global arena for a very short amount of training time thanks to the power of Vertex AI. 

To learn more

Battlesnake

https://play.battlesnake.com

Vertex AI – ML Training, Managed Pipelines, Hyperparameter Tuning, Model Serving

https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-launches-vertex-ai-unified-platform-for-mlops

https://cloud.google.com/vertex-ai/docs/pipelines

https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview

https://cloud.google.com/vertex-ai/docs/predictions/deploy-model-console

Machine Learning

https://learndigital.withgoogle.com/digitalgarage/course/machine-learning-basics

https://ai.googleblog.com/2021/04/evolving-reinforcement-learning.html

https://gym.openai.com/

Encryption is critical for securing sensitive data while it is stored and transits the cloud. Today, Cloud Storage encrypts data server-side with standard Google-managed encryption keys by default, and can also encrypt data with customer-managed encryption keys that are stored and managed by Cloud Key Management Service (Cloud KMS).

While customers have been able to secure their data on Cloud Storage with Cloud KMS keys for some time, we are always updating our encryption offerings to deliver better performance, lower costs and more capabilities that support critical business workloads. In this post, we’ll discuss some of our latest developments in this space—in particular, performance improvements for high-intensity workloads and support for customer-managed encryption keys (CMEK) for object composition. 

Why use Cloud KMS with Cloud Storage?

Cloud KMS enables you to centrally manage your keys in a fast and scalable way that helps to meet your security and compliance needs. Cloud KMS generates customer-managed encryption keys (CMEK) which act as an additional layer of protection on top of Google’s default encryption keys. You can set these keys on a Cloud Storage bucket as a default key, and can easily manage key rotation, replacement or disabling right within Cloud KMS.  

In addition to software-based CMEKs, Cloud Storage also supports hardware-based CMEKs hosted in hardware security modules that are FIPS 140-2 Level 3 validated as part of our Cloud HSM service. These enable you to protect your most sensitive workloads without needing to manage HSM cluster operations yourself. 

Improving KMS performance for high-intensity workloads

While Cloud HSM is often used to protect the most sensitive data for a customer, especially for those in healthcare and financial services industries, the default quota limits for cryptographic operations on Cloud HSM keys may cause performance bottlenecks for customers aiming to run high-intensity workloads when using Cloud Storage, such as analytics workloads on Hadoop.

We are making improvements to the Cloud KMS request behaviour on Cloud Storage that more effectively batches requests to Cloud KMS to reduce request bandwidth and drive down KMS billing. Identical Cloud KMS requests from Cloud Storage will be batched together for newly written objects and across all supported encryption modes in Cloud KMS, including software-backed customer-managed encryption keys. As a result of these changes, when you are using Cloud KMS you may notice faster encrypt, read and write operations for new data, deduplicated Cloud KMS audit logs and lower overall Cloud KMS charges. Customers running high-intensity workloads should see a significant reduction in throughput to KMS, leading to a reduction in KMS cryptographic operation billing costs for all types of KMS keys, as well as enabling customers to scale the throughput of their HSM-encrypted workloads.

Newly written objects encrypted with Cloud KMS will leverage these changes, whether they are encrypted using software-backed or HSM-backed CMEKs, and no configuration change is needed. If you are looking to use Cloud KMS with Cloud Storage, check out the Cloud KMS page to learn more, especially for setting up a hardware-backed HSM.

Supporting Cloud KMS for object composition

Object composition in Cloud Storage is widely used today for different types of applications, from stitching video segments together for a replay to uploading large datasets for analytics workloads. As more and more customers are leveraging Cloud Storage for these applications, we are expanding object composition capabilities to be flexible across different encryption modes.

Object composition is now supported for customer-managed encryption keys in addition to Google-managed encryption keys and customer-supplied encryption keys. This allows you to manage your own encryption keys while performing object composition for business-critical needs such as compiling sensitive financial datasets.

To compose objects that are encrypted with customer-managed encryption keys, specify the resource name of the Cloud KMS key for encrypting the composed object as a query parameter in the compose request. For the JSON API, construct the following HTTP request, while specifying the Cloud KMS key resource name for the query parameter kmsKeyName.

For the XML API, specify the Cloud KMS key resource name for the request header x-goog-encryption-kms-key-name.You can also specify a Cloud KMS key when using gsutil to perform object composition. Check out our documentation to try out object composition, or to start composing objects encrypted with customer-managed encryption keys.

Get started with better encryption

Being deliberate about encryption is critical for securing your sensitive data on Cloud Storage. Whether you are composing objects or running analytics workloads, leveraging the latest encryption offerings will deliver faster performance, better security and improved workload scalability. We’re always evolving our encryption products to meet your needs and help you achieve your business goals. To get started with encryption on Cloud Storage, check out our documentation to learn more.

This is the second part of a series of articles on using Anthos Config Management (ACM) with Terraform. In the previous article and example, we explained how to use Terraform to create a cluster and manage its configuration from Git via Config Sync. This article will build on that to add guardrails for the cluster via Policy Controller, a product within Anthos Config Management which enables the enforcement of fully programmable policies for your clusters. As we will see, these policies can audit and/or prevent changes to the configuration of your clusters to enforce security, operational, or compliance controls. 

In this case, we will enable an ongoing audit of Kubernetes cluster resources using the Policy Controller’s built-in Constraint Template Library as well as bundle of constraints enforcing CIS Kubernetes Benchmark v.1.5.1. Briefly, a Constraint Template provides the mechanism for describing policy declaratively and Policy Controller provides a built in library with Constraint Template implementations for some important best practices in Kubernetes. 

In the interest of simplicity, we will just be auditing the cluster via Policy Controller’s dryrun enforcement action. True to name, this is a “dry run” that only records violations and does not otherwise affect the cluster. However, once you are familiar with these constraints and how they would coexist with production workloads, you should consider using Policy Controller’s deny enforcement action which actively blocks config that violates policy from modifying the cluster. This gives you guardrails to prevent teams from drifting off road, automating centralized review to keep them moving safely.

If you take a look at the part2 example content, the main difference you will see relative to part1 occurs in the Terraform config which manages our cluster. Specifically, the following concise block is now added:

This is all that is needed to enable Policy Controller and also install the Constraint Template Library on your cluster. 

One other difference you may notice from part1 is the config-root/cis-k8s-1.5.1 directory which appears in the config root directory that is synced to the cluster via part1’s Config Sync. This is a bundle of Policy Controller constraints that were pulled into the repo from acm-policy-controller-library repo using the handy kpt tool.

The goal of this bundle is to audit and enforce CIS Benchmarks for Kubernetes. As mentioned above, the constraints have been deployed in dryrun mode so we can use them to audit the cluster.  We can inspect the respective Constraints directly to see any applicable violations, and in this example we’re using the also very handy tool jq to filter output to focus on the violation information:

This command groups the violations with the constraints that enforce them, and one such example is the following which warns abouts containers gaining elevated privileges:

Using Policy Controller, it’s clear that a vanilla WordPress installation was not really put together with security as a top level concern. In the example, you can see unnecessary privilege escalations, something that is all too common among applications. But with new support for Policy Controller in the Terraform Provider for Google Cloud Platform, it is now very simple to get audit and enforcement for best practices like CIS Benchmarks and the custom policies you require. 

This example repo provides a complete example and step by step instructions you can use to get started customizing and enforcing your own policies. We also encourage you to explore other best practices in the Policy Controller Constraint Library and this tutorial explains how you might go about validating apps against company policies in a CI pipeline.

As we learned in part1, this can all be driven via Git through Config Sync, a manageability plus that’s also a boon for attestation requirements of your security posture. In the next, and final part of the series, we will see how to use Config Connector to provision GCP cloud resources following the same Kubernetes-native model and tools.

If you need inexpensive managed messaging for streaming analytics, Pub/Sub Lite was made for you.  Lite can be as much as 10 times cheaper than Pub/Sub.  But, until now, the low price came with a lot more work. You had to manage the read and write throughput capacity of each partition of every topic.  Have 10 single-partition topics? Make sure you watch 10 write and another 10 read capacity utilization metrics or you might run out of capacity.

Hello, Reservations

We did not like this either. So we launched Pub/Sub Lite Reservations to manage throughput capacity for many topics with a single number.  A reservation is a regional pool of throughput capacity.  The capacity can be used interchangeably for read or write operations by any topic within the same project and region as the reservation. You can think of this as provisioning a cluster of machines and letting it handle the traffic.  Except instead of a cluster, there is just a single number. 

Less work is great, of course. It is even better when it saves you money.  Without reservations, you must provision each topic partition for the highest peaks in throughput.  Depending on how variable your traffic is this can mean that half or more of the provisioned capacity is unused most of the time. Unused, but not free. 

Reservations allow you to handle the same traffic spikes with less spare capacity.  Usually, the peaks in throughput are not perfectly correlated among topics. If so, the peaks in the aggregate throughput of a reservation are smaller, relative to the time average, than the peaks in individual topics.  This makes for less variable throughput and reduces the need for spare capacity. 

As a cost-saving bonus, reservations do away with the explicit minimum capacity per partition. There is still a limit on the number of partitions per reservation. With this limit, you pay for at least 1 MiB/s per topic partition.  This is not quite “scale to zero” of Pub/Sub, but beats the 4 MiB/s required for any partition without reservations.

An Illustration

Suppose you have three topics with traffic patterns that combine a diurnal rhythm with random surges.  The minimum capacity needed to accommodate this traffic is illustrated below.

“Before” you provision for the spikes in each topic independently. “After,” you aggregate the traffic, dampening most peaks.  In practice, you will provision more than shown here to anticipate the peaks you haven’t seen.  You will also likely have more topics. Both considerations increase the difference in favor of reservations. 

Are Reservations Always Best?

For all the benefits of shared capacity, it has the “noisy neighbor” problem.  A traffic peak on some topics can leave others without capacity.  This is a concern if your application critically depends on consistent latency and high availability.  In this case, you can isolate noisy topics in a separate reservation. In addition, you can limit the noise by setting throughput caps on individual topics. 

All in all, if you need to stream tens of megabytes per second at a low cost, Lite is now an even better option. 

Reservations are generally available to all Pub/Sub Lite users.  You can use reservations with existing topics. Start saving money and time by creating a Pub/Sub Lite reservation in the Cloud Console and let us know how it goes at pubsub-lite-helpline@google.com.

Contact centers are getting new tools in their digital transformation toolbelt. Starting today, Chrome OS offers a fully integrated solution for contact centers. With Chrome OS and our partners you get:

Secure platform and remote management: Chrome OS is a cloud-first, easy-to-manage operating system that can secure and optimize your contact center while supporting remote agents

Validated contact center solutions:New contact center solutions optimized for Chrome OS through a new solution track for Chrome Enterprise Recommended

Access to virtualization desktop infrastructure:Assurance knowing Chrome OS and Citrix Workspace is now a Citrix Ready verified solution for contact center

Agent peripherals & devices:New certified Works With Chromebook headsets and a thriving Chrome OS device ecosystem

A new world of customer engagement with Chrome OS

With work-from-home programs advancing at top speed — 77% of service organizations are adopting or accelerating remote work options according to Deloitte1 — more and more customer support agents are taking their calls from the home office. While businesses have adapted to this new way of work and quickly migrated many of their contact center solutions to the cloud, agent devices have been left behind. 

Chrome OS offers businesses a cloud-first OS for devices, remote management, and a productive agent experience that can secure and optimize contact centers: 

Protect business and customer information:Chrome OS has built-in and proactive protection from viruses, ransomware, and employee negligence. Security features like automatic security updates, verified boot, sandboxing technology, blocked executables, a read-only OS, the Titan C security chip, and Google Safe Browsing ensure data stays in the right hands. In fact, there have been no reported ransomware attacks on a Chrome OS device, ever.

Increase agent productivity: Agents can save up to 3 hours per week2 in downtime from updates that happen in the background, devices that don’t slow down overtime, and the cloud-first approach to collaboration and work. Additionally, the intuitive, error-free user experience means agents can onboard quickly and have a clutter-free desktop so they can focus on providing better customer support.

Deploy and manage stress-free:With zero-touch enrollment, IT can drop-ship devices directly to agents without IT configuration or imaging — users’ apps, settings, and policies download automatically. Chrome OS devices deploy 76% faster than Windows 10, and businesses can redeploy them 96% faster3 making dealing with high-turnover rates easier. The 500+ cloud-based policies allow IT to manage devices remotely from the Google Admin console.

Integrate with your preferred solutions:Deploy verified contact center solutions from the Chrome Enterprise Recommended program or integrate with your existing virtualization infrastructure to access on-prem solutions. Businesses can choose from a wide range of high-performance Chrome OS devices and certified peripherals for agents. 

Introducing the Contact Center solution track for Chrome Enterprise Recommended

Last year, we introduced Chrome Enterprise Recommended, our partner program featuring best of breed, third party software validated for Chrome OS by Google’s own partner engineering team. This program ensures leading software developers work with Google to optimize their products’ quality, security and end user experience, and provides businesses with a catalog of applications they can count on. To ensure contact centers can provide a great support experience, we are excited to announce the Contact Center solution track for Chrome Enterprise Recommended.

As part of this Chrome Enterprise Recommended solution track launch, we have worked with seven contact center solution providers to validate their market leading solutions, including 8×8, Cisco, Edify, Five9, Genesys, RingCentral, and Vonage. These solution providers have worked closely with our partner engineering team to optimize their products for Chrome OS and are driving innovation within the contact center industry.

We are excited about the improvements we’re bringing to our contact center customers in collaboration with Chrome Enterprise Recommended partners. Learn more about these new solutions, and check out their product demos here.

Chrome OS is now a Citrix Ready validated solution for contact center

Many contact centers still need to access legacy apps that exist on their infrastructure, and choose to access these applications through a virtual desktop infrastructure (VDI) solution, like Citrix Workspace, for business continuity or security reasons. Contact center leaders leveraging VDI need confidence their solutions will operate without interruption.

Today, we’re excited to announce Citrix Workspace on Chrome OS as a Citrix Ready validated solution for contact centers. This means that the Citrix and Chrome OS teams have validated leading contact center components including software, peripherals, and devices to work great together, ensuring contact centers get the best agent experience, simplified management, and shorter deployment times. For a complete list of what’s been validated, visit the Citrix Ready Marketplace page. Chrome OS and Citrix will continue to work together to validate more apps, endpoints, and peripherals. 

Together, Chrome OS and Citrix offer experiences designed for the end-user including strong product integrations that enhance audio and video passthrough, advanced security with built-in protection at the endpoint and in the cloud, and the ability to support any workload, from SaaS to legacy enterprise apps. 

New certified Works With Chromebook headsets and a thriving Chrome OS device ecosystem

For agents to provide superior customer service, they need the right tools to complement their devices including great mice, keyboards, monitors, webcams, docks, and headsets. The Works With Chromebook program certifies peripherals in all these categories to ensure seamless integration with Chrome OS devices. 

The program now includes 15+ certified headsets for contact centers from industry leading brands including EPOS, Jabra, and Poly. These headsets are tested & certified to work across all Chrome OS devices. See our device selectors to help you choose the best Chrome OS devices for your contact center and purchase verified peripherals here.

Start deploying Chrome OS today with customized solutions and offers

It’s easy to get started securing and optimizing your contact centers with Chrome OS today. You can explore customized solutions from CDW, deploy CloudReady to get a Chrome OS experience on your existing PCs and Macs, or start a Chrome Enterprise Upgrade trial if you already have Chrome OS devices. If you need Chrome OS devices, you can also contact us to get connected to a partner.

1From cost center to experience hub, Deloitte Digital, June 2021
22018 Chrome OS Forrester TEI Reports
3ESG Technical Review, Google Chromebook: Accelerating Device Lifecycle Management, June 2020

In this series of Recommendations AI deep dive blog posts, we started with an overview of Recommendations AIand then walked through the data ingestion process. In this post, we will walk you through the model creation process and how you can get the best of Google’s AI and ML techniques to solve your business problems without deep ML expertise. 

When you ingest the data into Recommendations AI, the product catalog and user events form the two key pillars to train the recommendation models. Product features such as Item id, title, categories, description, price, stock status, and Events such as homepage views, detail page views, add to cart, purchases depict the whole journey of each user. They enable Recommendations AI to capture the interconnection between a user’s history, current landing page, and next action, and build a deep neural network to provide personalized recommendations. Once the data successfully ingested, you are now ready to train your very first model. Woohoo!

Create recommendations in the graphical interface

Recommendations AI provides an easy-to-use graphical interface, with which you can choose your model type, define your optimization objective, customize your business rules, and begin training your model with just a few clicks.

The model will be trained for you from scratch using the data you sent to us within the bounds of the GCP project.​​ In 3-7 days the initial model training and tuning will conclude, and you’ll see a visual indicator in the dashboard that the model is ready to query. You can then begin serving recommendations to your customers, or visually preview the recommendations being generated by the model before you serve them to production traffic to ensure that your setup is satisfactory.

Google has spent years delivering recommended content across flagship properties such as Google Ads, Google Search, and YouTube. Recommendations AI draws on that experience to deliver personalized recommendations that suit each customer’s tastes and preferences across all your touchpoints.

Recommendation model types

Recommendations AI offers five recommendation model types including “Others you may like”, “Frequently bought together”, “Recommended for you”, “Similar items”, and “Recently Viewed“, with the first four model types powered by machine learning, and the new model type “Similar items” in preview.

The “Others you may like” recommendation predicts the next product that a user is most likely to engage or convert with. The prediction is based on the shopping and viewing history of the user and the candidate product’s relevance to a current specified product. In the example below, on a product detail page of a blue dress, the “Others you may like” model recommends another three dresses that the customer is likely to click on or purchase.

The “Frequently Bought Together” recommendation predicts items frequently bought together for a specific product within the same shopping session. If a list of products is being viewed, then it predicts items frequently bought with that product list. This recommendation is useful when the user has indicated an intent to purchase a particular product (or list of products) already, and you are looking to recommend complements (as opposed to substitutes). This recommendation is commonly displayed on the “add to cart” page, or on the “shopping cart” or “registry” pages (for shopping cart expansion). In the example below, the “Frequently Bought Together” model recommends heels and a watch, which go well with the blue dress just added to cart, and a green dress as the customer may need multiple dresses for different occasions.

The “Recommended for You” recommendation predicts the next product that a user is most likely to engage with or purchase, based on the shopping or viewing history of that user and contextual information of requests, such as timestamps. This recommendation is typically used on the home page. “Recommended for You” can also be useful on category pages. A category page is similar to a home page, except that you display only items from that category. You can achieve this using a standard Recommended for You model with filter tags. For example, you can add customized filter tags (corresponding to each category page) to the items in your catalog. When you send the prediction request, set the user event object as category-page-view and specify a specific category page’s tag in the `filter` field. Only recommendation results matching the requested filter tag are returned.

“Others You May Like” and “Recommended For You” are highly personalized recommendations powered by Transformer-based sequence modeling. “Others You May Like” recommendation requires an anchor item and is usually used on product detail pages, while “Recommended For You” can be used on pages without anchor items, such as home pages and category pages. They take into account both the content and order of a user’s historical clicks and purchases, correctly identify their true intentions, and precisely predict the next items that they would love to check out or purchase. For example, if a user browses a variety of maxi dresses and switches to fashionable shoes, “Others You May Like” model may recommend a mix of scandals and heels that go well with maxi dresses for a casual look during the day or a dressier look at night.

“Similar Items” is a new model type we’ve recently developed and is currently in preview. The “Similar Items” model will use only the product catalog, and not require user events in an effort to accelerate time to test model and preview recommendations results in the console for customers. We’ve leveraged self-supervised learning to accurately capture item similarities based on the metadata such as titles and categories, even if there is no user event ingested.

Optimization objectives

Machine learning models are created to optimize for a particular objective, which determines how the model is built. “Others You May Like” and “Recommended for You” recommendation models have click-through rate (CTR) as the default optimization objective. Optimizing for CTR emphasizes engagement, and you should optimize for CTR when you want to maximize the likelihood that the user interacts with the recommendation. In contrast, revenue per order is the default optimization objective for the “Frequently Bought Together” recommendation model type, as “Frequently Bought Together” focuses on cross-selling and increasing order values.

For “Others You May Like” and “Recommended for You” recommendation models, we also support conversion rate (CVR) as the objective. Optimizing for conversion rate maximizes the likelihood that the user adds the recommended item to their cart. When CVR is specified as the objective for a customer with sparse add-to-cart events, the multi-task learning mechanism will be automatically activated, and transfer learn from detail-page-view events, which are typically much denser than add-to-cart events.

User event data requirements

Before you create a new model, you must have met the requirements for creating a new model. The type of user events you import, and the amount of data you need, depends on your recommendation (model) type and your optimization objective. For example, you need to meet the following data requirement to train an “Others You May Like” model optimizing for click-through rate:

10,000 detail-page-view events that include at least 100 unique visitor IDs, and
1 week, with an average of 10 detail-page-view events per joined catalog item, or 60 days with at least one joined detail-page-view event.

When you reach the minimum data requirement, you can begin model training. You can import historical user event data to meet the minimum event data requirements faster, or wait until the user event data collection meets the minimum requirements.

Other features for better performance

Our models also support massive catalogs of tens of millions of items and ensure that your customers have the opportunity to discover the entire breadth of your catalog through personalized recommendations. Models are re-trained daily to draw insights from changing catalogs, user behavior, or shopping trends and incorporate them into the recommendations being served. We also correct for bias with extremely popular or on-sale items and better handle long-tail items with sparse data as well as seasonal items to ultimately drive better CTR, CVR and revenue lift for our customers.

Getting recommendations

The “Ready to Query” column on the Models page will change to “Yes” once the models are finished with the initial training and tuning.  Once your models are trained and ready to query, you can preview the results in the cloud console and make prediction requests with the API.  Requests aren’t made to a model directly, a predict request is made to a specific Serving Config (previously called a Placement).  A Serving Config contains some additional options to use – so you can have multiple placements that call the same model, each with different options like price reranking or diversification settings.

Creating a Serving Config:

Name your Serving Config

Choose a model

Select Options

Usually you’ll want to create a separate serving config for each different placement of a model.  This is useful for reporting, and also allows you to change the model type in one particular place on a site without affecting other placements.

Recommendations AI is migrating to a general Retail UI that can be used for both recommendations and search, and some options will change in the new Retail UI. For example, we will be supporting the configuration of business rules in serving configs instead of models, so that you can quickly try out serving configs with various business rules, such as different levels of price reranking or diversification, using the same underlying model.

The Evaluate (preview) tab allows you to quickly see what the actual results look like.  The required parameters like visitorId or item ids can be entered in the form and then submitted to get results from a particular Serving Config/Model combination:

You can leverage Recommendations AI anywhere in the customer journey.  Home page, product detail pages and cart pages are the most common placements, but you can also include recommendations in email campaigns and mobile apps. Let’s take a look at the customer journey here as an example.

A customer begins their journey on their laptop, browsing the sports apparel category when Recs AI recommends shoes. The customer leaves, but comes back to the site on their phone to view more shoes. Recs AI steps in again to recommend sunglasses. Our customer adds sunglasses and shoes to their cart, indicating an interest in both items. Here, Recs AI recommends shorts which the customer clicks on to look at, but ultimately abandons their cart.

Three days later, you can send a personalized follow-up email to the customer with recommended items based on the customers clicks and interests. Recs AI closes the loop, spurring the customer to buy sunglasses and shorts, both recommended based on the customer’s previous activity across devices.

As we can see, Rec AI works throughout the entire buying process – from initial product discovery on a laptop, to consideration on a mobile device, to purchasing follow-through via email channels.

And that’s just one example of delivering recommendations in the customer journey. Customers can use recommendations across different device types (mobile, computer), in email campaigns, in physical retail settings (kiosks), or even indirectly (call center representative, sales representative).

If you’ve been following along the steps with these blog posts series, you would now have a ML model that is trained for your custom data and ready to query. In our next post, we will go over how to include the Recommendations from these models to incorporate on your site. 

BigQuery is a fully-managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and intelligent caching for business intelligence. To help you make the most of BigQuery, we’re offering the following no cost, on-demand training opportunities: 

BigQuery basics

In this training, you’ll learn the basics of BigQuery like how to use the Web UI to query public tables, and how to load sample data into BigQuery from a Cloud Storage Bucket. Get started here. 

How to use machine learning with BigQuery  

Find out how BigQuery leverages machine learning to automate data quality rules – scanning disparate and dynamic datasets, detecting anomalies, discovering sensitive data – all of which help you improve the quality, accessibility, and governance of your data. Sign up here for this training.

Discover how to use BigQueryML to build forecasting models at scale, plus learn how to schedule model re-training on a regular basis so that your forecasts stay up-to-date when you register here for this training. 

Sign up here to find out how to create an end-to-end solution for real-time fraud detection with the help of BigQueryML. 

How to connect BigQuery to Looker 

Learn how to model, analyze, and visualize your data in less than 30 minutes when you register here for this training. You’ll be taught how to set up a connection between your BigQuery data warehouse and Looker. Once connected, you’ll learn how Looker auto-generates a data model, which you can immediately explore, customize, and reuse. From there, you’ll discover how to build reports, dashboards, plus bring data into your workflows via alerts, integrations, and more. 

For more demos on how to use BigQuery and Looker together, sign up here. 

What’s next with BigQuery? 

To get a peak into product roadmap enhancements and hear from our Product Management Director, Jonathan Keller, about what’s next with BigQuery, register for Next ‘21 and join us for this session on October 12.

If you are looking for an automated way to manage quotas over a large number of projects, we are excited to introduce a Quota Monitoring Solution from Google Cloud Professional Services. 

By default, Google Cloud employs resource quotas to restrict how much of a particular shared Google Cloud resource you can use. Each quota represents a specific countable resource, such as API calls to a particular service or the number of compute cores used concurrently by your project.

Quotas are enforced for a variety of reasons, including:

To protect the community of Google Cloud users by preventing unforeseen spikes in usage and overloaded services.

To help you manage resources. For example, you can set your own limits on service usage while developing and testing your applications to avoid unexpected bills from using expensive resources.

Quota Monitoring Solution solution benefits anyone who manages quotas across projects, folders, or organizations. It offers an easy and centralized way to view and monitor the quota usage in a central dashboard and to use default alerting capabilities across all quotas. 

Specifically, the solution provides:

Automated aggregation of quotas across all projects in given organizations or folders and a recurring scan at a defined frequency (e.g. hourly, daily) for new projects to automatically capture their quotas.

A dashboard that provides visibility into recent resource usage against the individual quotas across all projects.

Preconfigured alerting through email or other communication channels (e.g., email, SMS, Pub/Sub, etc.) when a resource reaches a certain threshold of its quota.

The solution is easily deployable through Terraform so that you can adopt it into your project with minimal time investment.

Outside of the Quota Monitoring Solution, there are additional ways of viewing your quota information, such as using the Google Cloud Console or using the gcloud command-line tool. You can also manually define Alerting Policies in Cloud Monitoring to send out notifications when a resource reaches a certain threshold of its quota. For example, you can define an alerting policy that triggers when the CPU usage of Compute Engine VM instances goes above 75% of the quota in any region.

In case your project needs more of a particular resource than your quota allows, you can request a quota limit increase for the majority of quotas directly in the Google Cloud Console. In the vast majority of cases, quota increase requests are evaluated and processed automatically. However, depending on the nature of your request, a small number of quota increase requests needs to be handled by human reviewers. They typically process your request within 2-3 business days, so it is important to plan ahead.

Ineffective quota management can lead to many different problems. For example, the lack of sufficient quota can prevent consuming additional resources, which could be needed for auto-scaling events, or for performing a GKE cluster upgrade. This can cause outages or service degradation, which could impact your customers’ experience and potentially impact your business revenues.

Please note: Many services also have limits that are unrelated to the quota system. These are fixed constraints, such as maximum file sizes or database schema limitations, which cannot be increased or decreased. You can find out about these on the relevant service’s Quotas and limits page (for example, Cloud Storage quotas and limits).

1. Technical Architecture

The diagram below shows the Quota Monitoring Solution architecture flow you can deploy in minutes using the deployment guide and accompanying terraform scripts.

The solution includes a terraform script that you can deploy in a GCP project. This templates provisions the following resources in a GCP project:

Cloud Scheduler– Cloud Scheduler is a fully managed, enterprise-grade cron-job scheduler. It is used to trigger Cloud Functions at scheduled intervals.

Cloud Functions – Cloud Functions is an event-driven serverless compute platform. It contains the code logic to scan project quotas.

Pub/Sub– Pub/Sub allows services to communicate asynchronously, with very low latency. It is used to support event-driven application design and high scalability. 

BigQuery –  BigQuery is a serverless, highly scalable data warehouse. It is used to store data for project quotas.

Data Studio – Data Studio is a Dashboard and reporting tool. It is used to display quotas across multiple projects in a single view. You can configure other visualization tools of your choice, like Looker. In the future, we will also provide a Looker-based Dashboard.

Cloud Monitoring Custom Log Metric and Alert – Google Cloud Monitoring offers logging and alerting capabilities. This is used to enable alerting that gives timely awareness to quota issues in your cloud applications so you can request a quota increase quickly. 

In this solution, Cloud Scheduler also works as an interface for you to provide configurations:

You can provide folder IDs, organization IDs as parent nodes for which you want to monitor quotas. Parent nodes can be a single folder ID or organization ID or list of folder IDs or Organization IDs. 

You can also configure metric threshold and email addresses for Alerting. 

Currently, you can receive alerts via Email, Mobile App, PagerDuty, SMS, Slack, Web Hooks and Pub/Sub.

Metric threshold is used to generate alerts. For example you can choose 70% or 80% as a threshold to generate alerts. 

The alerts will be sent to the configure alerting channel. 

Please note:

Any changes in the projects like addition of new projects or deletion of existing projects will be reflected automatically in the subsequent scheduled scanning.

Any changes in the GCP Cloud Monitoring APIs will be reflected. For example,  the introduction of new quota metrics will be reflected automatically in the solution without making any code changes.

Any changes in the Cloud Monitoring alert notification channels will also be available automatically without the need to make any code changes.

1.1 Workflow

If we put all these components together, the workflow starts at Cloud Scheduler. You can provide your preferences of folders and organization IDs, metric threshold, email addresses and configure to run the job at scheduled intervals. 

Create a service account and grant access to view quota usage in the target organization/s or folder/s.

Cloud scheduler automatically runs at configured frequency, for example daily, and passes user configurations to Cloud Functions. 

Based on the service account access, the first Cloud Functions instance lists projects of the parent node, generates the list of project IDs, and publishes them to the Pub/Sub topic.

The second Cloud Function instance is triggered from the message generated in the previous step and receives the project IDs as a separate message from the topic.

Upon receiving the project IDs, the second Cloud Function instance fetches the quotas for each project using publicly available GCP cloud monitoring APIs. 

Cloud Function loads the project’s quota data in the BigQuery table. 

Alerting 

A scheduled query on the BigQuery table filters all quotas with usage greater than the configured threshold.

The third Cloud Function’s instance logs the data for metrics in Cloud Logging.

Preconfigured custom log metrics in Google Cloud Monitoring create alerts, which are sent to the configured notification channel. 

Reporting

Once the data is loaded into BigQuery, Data Studio fetches the data from BigQuery and displays it in the Dashboard in a single view across all projects and metrics.

Data Studio Dashboard is easy to use, customize, and share. 

You can also configure scheduled reporting from this Dashboard to receive Quota monitoring reports in email as PDF.

2. Deployment Process

The Google Cloud custom quota monitoring solution can be deployed in a few minutes using the deployment guide and accompanying terraform scripts. Following are three simple steps:

Create a service account and grant access

Run terraform script to provision all resources in a Google Cloud project

Configure the Data Studio Dashboard 

For step-by-step details, please see the Readme section of the deployment guide on PSO Github. 

3. How to Customize?

This is an open source code available on Google Cloud PSO Github repository. You can fork the repository and customize as per your requirements. During the deployment, Terraform downloads the code. The code and the data stays in the customer’s environment. 

4. Summary

Quota Monitoring solution lets you to view and monitor quotas for Google Cloud services from one central location for an organization or folder. Service Quotas enables you to easily and consistently track quota usage and receive alerts to save time and effort in setting up quota monitoring for new projects.

If you have questions or suggestions for this solution, please fill out this form or reach out to us at pso-quota-monitoring@google.com 

Acknowledgement

We would like to thank Darpan Shah, Raghavendra Kyatham, and Marlon Pimentel for their contribution in building the solution as well as Rahul Khandkar, Karina Rivera, and Rohan Karne for sponsoring this project. We would also like to extend our gratitude to all Technical Account Managers who helped prototype and roll out the solution globally, especially Naveen Nelapudi, Vijay Narayanan, Rohit Iyer, Emily Wong, and Isha Rana.