Cloud

At Google Cloud, we know that developer time is precious. That’s why we’re always looking for ways to improve productivity. We’d like to share ten new features that we launched in Cloud Logging so that you can more quickly get to the logs that matter and resolve issues more quickly. 

Improved logs display for Google Kubernetes Engine (GKE) in the Cloud Console  – improved display and log filtering in the GKE section of the Cloud Console

Pivot around a log entry – easily find related logs by pinning a log entry

Filter severity updates  – more easily filter by severity and above to find error logs

Histogram filtering – use the histogram to easily filter logs

Hide/Show similar log entries – reduce noise by hiding logs 

New date/time options – new quick options to make date/time selection easier

New SEARCH function + indexes – a new built-in function to find strings in your logs that uses indexes to reduce query time

Search/highlight in log results – find values in log results 

Copy/paste log improvements – better copy/paste formatting

Error Reporting updates – filter by GKE resources in Error Reporting and integration into the GKE section of the Cloud Console 

In this blog post, we’ll focus on the actions that happen after an issue has been identified with an alert or dashboard. Using an example, we’ll step through the process of troubleshooting an example application and point out how the new features streamline the troubleshooting steps. Since there are too many features to cover in a single blog, we’ll talk about the first five here and the others in follow-on blog posts.

Better GKE troubleshooting, by example 

Here’s an example scenario and how the new features help reduce time spent finding logs. In the scenario, there is a microservices-based ecommerce application that is deployed to GKE. Each microservice generates logs which can be used to understand the system behavior. There are intermittent errors generated in our application that have prompted a troubleshooting session to understand the root cause. 

First up, let’s take a look at the service in GKE. In GKE, metrics and logs are integrated into the workload and cluster pages via the observability and logs tabs. You can find your infrastructure metrics and application logs right from GKE! The screenshot below shows the error logs for the frontend service.

1. A new look for the GKE logs tab 

With this latest launch, when the logs are expanded, you can see the full JSON structure and all the log fields which can help you quickly see the details of the errors. Basic filtering by severity and text search helps to narrow to specific logs. When you need more filtering capabilities and a histogram, there is a link to open Logs Explorer with a fully qualified search for the container logs.

A key part of troubleshooting is often finding a specific log of interest and then looking for the context around that log. Pivoting around a log entry can help to better understand the context of the logs. For example, looking at the full set of logs for a specific request that generated an error can help better understand the system behavior.

2. Pivot around a log entry

Under the dropdown next to the pin icon on each log entry, there are now two options to find related logs for a specific log: 

Same resource.type – shows all the logs in the same resource type, k8s_container in this example 

Same resource.labels – shows all the logs generated by the same resource, the frontend container in this example

In the example, using the “Same resource.labels” shows the error logs in the context of all the logs generated by the container. The specific log entry is also pinned, which means that the pinned log entry will stay at the top of the list of logs for easy comparison to other log entries. Now, it’s easy to see that while there are errors, there are also many successful requests and now the details of each request are easy to see.

Severity is a key attribute in logs and sometimes it’s useful to filter in or out logs based on their severity. With this recent change, the severity selector can be used to filter out debug logs or even show logs with error and higher severity. 

3. Filter by severity from log line

In our example, using the “Hide debug entries” hides all of the logs after they’ve already provided the necessary context.

This same feature is also now available in the severity dropdown.

One key challenge in troubleshooting is finding the signal amidst the noise. Now, with the “Hide similar entries” feature, Cloud Logging gets rid of the noise for you. 

4. Hide similar logs

In the example, the “Hide similar entries” is displayed using a purple row in the log results, which includes an option to filter out the noisy logs. Selecting “Hide similar entries” applies the right filter to your query to remove the logs. Using the Preview option, the specific details about the query and the sample logs are displayed.

The same ability to filter is also available in each log entry. With this recent launch, you can now Hide, Show or Preview similar log entries to help filter out noisy logs by adding a query filter.

Understanding the frequency of an error or event in the logs is an important aspect of troubleshooting. The histogram in the Logs Explorer provides several simple ways to help navigate to the important logs. 

5. Using the histogram to filter logs

Expand histogram – when you need to dig into the frequency, expand the histogram to make it larger

Synchronized scrolling – as you scroll through the logs, the histogram shows the corresponding time window on the histogram

Zoom in/out – the zoom in/out expands or decreases the time range of the logs

Scroll/zoom to time – query or scroll to the logs for a specific time range

In the example, narrowing the time range to focus on the logs around a spike in error logs helps narrow down the logs to a specific configuration issue. With this context, enough information is available to update the configuration and resolve the issue. 

What’s next

These new features in Cloud Logging can help you find the critical logs even faster. But we’re not stopping there. This is the first post in a series about the new features specifically designed by developers for developer troubleshooting in Cloud Ops. In our second post in this series, we’ll highlight other recent improvements to Logging and Error Reporting. In the meantime, if you haven’t tried Logs Explorer, you can get started today.

Firewalls are a critical component of your security architecture. With the increased migration of workloads to cloud environments, more companies are turning to cloud-first solutions for their network security needs. 

Google Cloud Firewall is a scalable, cloud-first service with advanced protection capabilities that helps enhance and simplify your network security posture. Google Cloud Firewall’s fully distributed architecture automatically applies pervasive policy coverage to workloads wherever they are deployed in Google Cloud. Stateful inspection enforcement of firewall policies occurs at each virtual machine (VM) instance. 

Cloud Firewall offers the following benefits: 

Built-in scalability: With Cloud Firewall, the firewall policy accompanies each workload as part of the forwarding fabric, which enables the service to scale intrinsically. This can relieve customers of the operational burden to spend time and resources to help ensure scalability. 

Availability: Cloud Firewall policies automatically apply to workloads wherever they are instantiated in the Google Cloud environment. The fully distributed architecture can allow for precise rule enforcement, even down to a single VM interface.  

Simplified management: Cloud Firewall security policies for each workload are independent of the network architecture, subnets and routing configuration. The context-aware and dynamically updating objects for firewall rules enable simplified configuration, deployment and ongoing maintenance.

How to migrate from on-prem to Cloud Firewall 

Most on-premises firewall appliances, either virtual or physical, are deployed in one of two modes: 

Zone-based that creates trusted and untrusted zones to apply firewall policies; or 

Access Control Lists (ACL) applied to an interface. 

In both cases, the firewall’s primary purpose is to protect one perimeter or network segment from another. For example, you may use a zone based firewall to filter traffic from an “untrusted” to a “trusted” zone. Similarly, you may have an ACL-based firewall to protect an “inside” network segment from an “outside” network segment.  

However, that strategy is not the best approach with Google Cloud Firewall policies and rules. Cloud Firewall is not designed to act as a perimeter device; rather, Cloud Firewall is a fully distributed set of rules to help protect individual resources, such as VMs. However, most of our customers want to replicate their on-prem firewall logic and apply it to their cloud environment.  Take the following example:

There are a lot of similar components shared between on-prem firewall appliance rules and Cloud Firewall rules. However, some critical differences between them can make a migration from firewall appliances to Cloud Firewalls a challenging task, for example:

Traditional firewalls protect a perimeter. In Google Cloud, firewall rules protect resources. This is done through the concept of “targets,” which specify which resources a given firewall rule applies to.

There are multiple types of firewall options available in Google Cloud (hierarchical firewall policies, global/regional firewall policies, and Virtual Private Cloud (VPC) firewall rules). Deciding which type of rules to use, and how to configure the rules with your cloud network architecture requires review and planning. 

Furthermore, there are some additional firewall rules that may be needed in a cloud environment when compared to an on-prem firewall. For example, you may need to create ingress firewall rules to allow Google Cloud health check traffic to load balancer backends or you may need to create an egress rule to allow VMs access to use the Google Cloud APIs. Further, on-prem firewalls often have additional functions in on-prem networks including routing, NATing, VPN termination, and in some cases, Layer 7 inspection.

To assist customers with the migration from on-prem firewall appliances to Cloud services, including Cloud Firewall, we have developed a best practice guide that includes design and architecture considerations, and a side-by-side comparison of on-prem to Cloud Firewall rules. Check out the guide here for more information.

Editor’s note: On a trip to visit family in Iran, Sophia Kianni made an alarming observation. Despite facing the disproportionate impacts of climate change, and with temperatures in the Middle East rising twice as fast as the global average, her relatives knew almost nothing about the world’s environmental challenges. When she realized that scientific literature wasn’t readily available in Farsi, she took it on herself to translate crucial documents, helping her relatives understand the threats of climate change and the opportunities for action. This inspired her to establish Climate Cardinals, which makes climate resources more accessible to non-English speakers. In this blog post, Sophia and her colleague Hikaru Hayakawa from Climate Cardinals talk about the importance of accessible climate education and the empowering potential of AI technologies in the fight against climate change.

How can you help stop a crisis you don’t know is happening? 

Education is key to climate action, but there’s a barrier that continues to lock billions of people out of the conversation: language. For too long, the international climate movement has been inaccessible to people who don’t speak English. We’re talking about 80% of the world’s population. 

Even though only one in five people speak English, more than three-quarters of scientific publications are only ever released in English. Even reports from the Intergovernmental Panel on Climate Change (IPCC), which contain the most comprehensive assessments of how to reduce the impact of climate change, are only available in the six official UN languages. This still leaves out approximately half of the world’s population. 

We saw an opportunity to change this. In the first three months of using Translation Hub, with support from the Google Cloud team, we were able to deliver the same volume of translations we had produced in our first two years of operation. This included producing a synthesis of the IPCC report and translating it into over 25 languages.

Climate change is the biggest issue of our generation, so we need to make sure that people have the resources they need to make sense of the disasters that are destroying their communities. At Climate Cardinals, we’re focusing on translation as a vehicle for disseminating this crucial information to the communities most impacted by climate change.

Mobilizing everyone, everywhere to fight against climate change

Climate Cardinals launched during the pandemic, immediately earning more than 1,000 volunteers for our cause on the first day. From the start, it’s been a truly youth-led movement: the average age of our volunteers is 16, and we’ve been banging the drum on social media to recruit more young people and expand our executive board.

The enthusiasm of our volunteers keeps our climate mission moving. In our first years of operation, we managed to translate more than 500,000 words of crucial climate information, like climate glossaries and UN documents, into more than 40 languages. We were also able to form partnerships with UN organizations and professional translation networks, like Respond Crisis Translation or Translators Without Borders, for editing and proofreading, helping us to ensure that our translations are correct and credible. 

But there have also been challenges. Because our capacity depends on the busy schedules of our student volunteers, we had to be selective about the translation work we could accept. And as students ourselves, managing the high interest and workflows of thousands of volunteers meant constantly adapting and refining our approach, all while staying on top of our other work. So when the Google Cloud team reached out to us, halfway through our third year of operation, we instantly recognized the potential to elevate our efforts and make a greater impact by joining forces.

Accelerating climate action with AI

Initially, a Google Cloud team offered to lend us capacity as part of their social impact initiative. This was a multilingual group in the EMEA marketing team in Google Cloud, who volunteered their time as part of the GoogleServe program. Soon after conversations started, the Google Cloud team saw potential for automation using their AI tools, and started an internal trial of the newly launched Translation Hub product. 

Translation Hub is Google Cloud’s AI-powered self-serve translation platform, and it was a real game changer.

Translation Hub automatically translates documents with AI, and then lets us augment and edit the results with human translators. The Hub can process PDFs with design elements and returns a translated document with the same design. It can also translate text within design elements. Initially, we worked hand in hand with the Google Cloud team to integrate the solution into our processes. When we had new translations, we sent them over to the Google Cloud team, who ran them through the Translation Hub. The finished translations then went back to the volunteers in addition to the Google Cloud team, who would proofread, edit, and polish them.

The change in pace was immediate. In the first three months of using Translation Hub, we translated 500,000 words of vital climate information for non-English speakers, for whom there is a drastic asymmetry in available educational materials. But that was only the beginning.

Amplifying climate outreach with Translation Hub

Today, we’re using Translation Hub all by ourselves, and it has become an essential part of our translation pipeline. As a result, we’re more productive than ever. No longer held back by capacity issues, we’ve been able to take on larger, more ambitious projects. That includes documents as long as 200 pages, which would have been tough to manage in the past.

For example, we translated around 100,000 words of climate information into Spanish for Yale Climate Connections to increase their outreach among Latinx communities in the U.S. We also translated documents into more than 20 languages for various global organizations to increase their outreach among youth. These organizations include the United Nations Sustainable Development Network, the Climate Mental Health Network, the United Nations Association Climate and Youth Council, the Environmental Justice Foundation, and the United Nations Committee on the Rights of the Child.

Our busy volunteers really appreciate the helping hand from AI, and so do our partners. According to Yale Climate Connections Editor-in-Chief Sara Peach, “Our partnership with Climate Cardinals has super-charged our ability to make crucial weather and climate change information accessible to our Spanish-speaking audiences.”

Translation Hub has made it much easier for our team to manage incoming translation requests and allocate tasks to proofreaders. The process is now more streamlined, allowing us to be more efficient and conscious of our volunteers’ time and capacity. They can better juggle their volunteer work with school and other commitments, including other forms of climate activism.

Making climate information a public good

As we continue to break language barriers and spread vital climate change information, it’s essential to remember that translation isn’t an end in itself. Our work gets its meaning from its potential to empower and educate people around the world. And while climate change impacts all of us, it has a disproportionate effect on communities that are often left out of the conversation. That’s why it’s crucial to ensure that climate information becomes a public good, accessible to everyone, regardless of language or background. We believe that AI can help make that happen.

With the power of AI, we can create a world in which language barriers in climate information are a thing of the past. We couldn’t do this without the technical expertise of Google Cloud. And Google Cloud couldn’t do it without the passion, community engagement, and grassroots connections of Climate Cardinals. That’s what makes our joint project truly unique. It’s an international, intergenerational, multilingual collaboration between the public and private sector.

Together, we’re offering tech solutions that can bring climate knowledge to the global stage, fostering understanding and inspiring action for a more sustainable future. We hope it’s a model for other youth organizations who are looking for innovative ways to tackle climate change.

In our conversations with customers, few generative AI use cases have driven as much enthusiasm as generative search. Leaders at enterprises know the limits of traditional enterprise search, with queries producing a list of links based on pattern matching, and significant manual investigation required to find the more relevant answers. In generative AI, these leaders see an opportunity to leverage their data more effectively and deeply, including applying it to conversational apps that can answer complex questions, produce accurate summaries that synthesize many sources, and help people get the information they need, faster. 

Enterprise Search on Generative AI App Builder (Gen App Builder) lets organizations create custom chatbots and semantic search applications in as little as a few minutes, with minimal coding needed to get started and enterprise-grade management and security built in. Customers can combine their internal data with the power of Google’s search technologies and generative foundation models, delivering relevant, personalized search experiences for enterprise applications or consumer-facing websites. 

To date, most approaches to combining generative AI and search technology have been inadequate for the scale and reliability needed for enterprise use. For example, building search by breaking long documents into chunks and feeding each segment into an AI assistant typically isn’t scalable and doesn’t effectively provide insights across multiple sources. Likewise, many solutions are limited in the data types they can handle, prone to errors, and susceptible to data leakage. Bespoke, do-it-yourself approaches are generally no easier, with production-grade solutions often requiring complex tasks like integrating embeddings data with foundation models and significant use case data testing. Even when organizations make these efforts, the resulting solutions still tend to lack feature completeness and reliability, with significant investments of time and resources required to achieve high-quality results.

These challenges demonstrate that to effectively implement generative search, organizations typically need more than access to powerful foundation models. They also likely need the ability to ground model outputs in specific data, so that outputs are more relevant and less likely to include mistakes or “hallucinations.” They generally need safeguards that protect their data, how it is accessed, and how it is used. And they generally need the process to be high-performant and scalable out of the box, making the functionality easy to use even if the organization lacks data science and machine learning expertise.

Let’s look at how Enterprise Search on Gen App Builder helps customers bypass these scale and reliability challenges, so they can start leveraging generative search quickly. 

The intersection of generative AI and enterprise data 

Enterprise Search on Gen App Builder lets developers create search engines that help ground outputs in specific data sources for accuracy and relevance, can handle multimodal data such as images, and include controls over how answer summaries are generated. Multi-turn conversations are supported so that users can ask follow up questions as they peruse outputs, and customers have control over their data—including the ability to support HIPAA compliance for healthcare use cases. All of this is available as a fully managed service, so developers can focus on building rather than cloud complexity. 

Gen App Builder’s out-of-box capabilities can remove the need for data chunking, generating embeddings, or managing indexes, hiding complexity behind a straightforward interface that lets developers build apps in minutes with little or no coding and no prior machine learning experience. With the ability to ingest large volumes of documents and support for both unstructured and structured data, apps built with Gen App Builder help customers solve the long-standing headache of finding relevant information across the organization, turning tasks that used to take hours into quick searches or conversational explorations with an app. 

These capabilities are underpinned by both Google’s foundation models and a variety of Google Search technologies, including:

Semantic search, which helps deliver more relevant results than traditional keyword-based search techniques by using natural language processing and machine learning techniques to infer relationships within the content and intent from the user’s query input. 

Google’s understanding of how users search for information. 

Google’s expertise in understanding relevance, which considers factors such as content popularity and user personalization of content when determining the order in which search results are displayed. 

For more advanced use cases, Gen App Builder can be easily integrated with Vertex AI for in-depth foundation model tuning, enabling flexible input and output search formats. 

As always with our AI products, Gen App Builder has been evaluated for alignment with our AI Principles, which is reflected by the product’s many safeguards against bias, toxic content, and unhelpful outputs. Whether for prototypes built in minutes or apps with many custom components, Enterprise Search on Gen App Builder offers a robust suite of user-friendly tools for customers across industries and levels of expertise. 

How customers are innovating with Enterprise Search 

After gaining access to Enterprise Search on Gen App Builder via our trusted tester program, a number of customers are already leveraging the product for novel use cases. 

Priceline is harnessing Gen App Builder and Vertex AI for a range of projects, including internal search engines for employees and a new chatbot to assist customers as they make travel plans. Slated to be available across both desktop and mobile experiences, Priceline’s chatbot will help customers find the right information faster via always-on, personalized experiences, including answering nuanced questions like, “What are the best 4-star hotel options in midtown Manhattan within walking distance to Central Park?” and “Can you help me extend my hotel reservation for an additional night?”

“Priceline is charting a course to transform the novelty of generative AI into lasting value for our customers and our business. We believe it’s not just about having the latest technology; it’s also about practically targeting innovation to the right challenges and opportunities,” said Marty Brodbeck, Chief Technology Officer, Priceline. “With Google Cloud as our AI innovation partner, we’re doubling down on our commitment to delivering the fastest, most seamless and informative booking experience for our customers, from personalized planning and travel inspiration to customer service.”

Vodafone is experimenting with Enterprise Search and foundation models on Gen App Builder to build a tool that can rapidly and securely query documents, search, and understand specific commercial terms and conditions. Vodafone Voice and Roaming Services has over 10,000 contracts with other telecommunications companies worldwide, in a variety of formats such as PDFs, images, and complex tables. Searching this document repository is often a time consuming process for employees. 

“Every day we introduce and manage new services like 5G to a roaming footprint comprising more than 700 operators in 210 countries to ensure both Vodafone and our partners’ business customers, holidaymakers, and Internet of Things devices stay connected when abroad. With Enterprise Search on Gen App Builder, we are building an intelligent assistant to securely and quickly search contracts.” said Sherif Bakir, CEO of Vodafone Voice and Roaming Services. “With generative AI, we’re accelerating otherwise protracted processes, increasing productivity and operational efficiency.”

Software startup Trender.ai, which creates customer monitoring and intelligence solutions for B2B relationships, is using Gen App Builder to build a product that can synthesize information from social media, public sources, and CRM data so that users can form more productive, personal relationships with prospects and customers.

“With Enterprise Search on Gen App Builder, we have been able to do things in the last month that we had projected to take 12-18 months on our prior roadmap,” said Betsy Bilhorn, co-founder at Trender.ai. “We had expected it would take at least 12 months to build and train models from an individual’s public social, web, and other data, and to then be able to ask questions like ‘What things are most important to this person?’ or ‘When and how is the best way to make an introduction to this prospect that they’ll respond to?’ Last year, we thought getting to this vision was a bit of a moonshot for a startup our size — but now we were able to achieve this in under a month.” 

Stop searching for solutions — start discovering insights

We’re excited to see how customers use Enterprise Search on Gen App Builder to leverage data in powerful ways, discover new insights, and create useful, personal, and efficient experiences. 

Today, we’re pleased to help our customers accomplish these goals, with the general availability of Enterprise Search on Gen App Builder for customers on the allowlist (i.e., approved for access). Please contact your Google Cloud sales team for access and pricing details. We are also launching two new features within Enterprise Search on Gen App Builder that are available today in our preview offering: multi-turn search, which supports asking follow-up questions, and content recommendations to find semantically relevant content. Access to the preview offering is available via Google Cloud’s trusted tester program. Read more about Enterprise Search on Gen App Builder and sign up for access on our webpage. 

We’re also bringing generative AI features of Enterprise Search to our existing solutions like Contact Center AI and Document AI. As an example, starting this month, customers can preview generative AI-powered search in Document AI Warehouse. To keep up with our latest generative AI news, don’t miss The Prompt or our generative AI primer for executives on Transform with Google Cloud.

What will financial services look like in the future? Historically, financial services have been driven by one-on-one relationship-based interactions. 

Now, customers expect these trusted relationships to translate into personalized, digital experiences that prove their providers know and value them, and have the capabilities to help them.

It’s no secret that customer engagement leads to revenue growth. That’s why customer data platforms (CDPs) have gained traction across multiple industries to help businesses analyze, predict, and personalize customer journeys.

However, a CDP by itself may not be sufficient to gain a holistic view of the customer. Financial services institutions already capture a myriad of customer and marketing data and use many tools to engage their customers. But companies still struggle to unify and analyze all that data across silos and create a holistic view of their customers that’s actionable across different channels and capabilities.

Financial services companies will need to think strategically about maximizing their use of first-party data and analytics to drive better and sustainable business outcomes.

With a true 360 view of the customer, financial services institutions can seamlessly combine all the right data and gain meaningful insights to deliver the type of personalization that really matters to customers.

Google Cloud is here to help

Customer expectations are changing rapidly, and financial services leaders need to be as innovative and efficient as possible. By combining customer servicing and transactional data with marketing, channel, CRM, and other first-party data within a secure cloud-based environment, equipped with intelligent, ML-driven customer insights, financial services companies can uncover deeper customer insights and activate them quickly.

To learn more or explore how Google Cloud fits into your customer data technology strategy, please contact us or visit our website.

Footnotes

1. “The value of getting personalization right—or wrong—is multiplying,” McKinsey & Company, November 12, 2021
2.  “Survey shows customers give banks lackluster grades on their personalization,” Insider Intelligence, April 11, 2022
3.“The Future of Insurance: Personalized, Digitized and Connected,” Capco Insurance, September 21, 2021
4.Press Release: “72% of Consumers Would Share Personal Data to Get Cheaper Insurance Premiums,” Capco, July 22, 2021 
5. “The value of getting personalization right—or wrong—is multiplying,” McKinsey & Company, November 12, 2021
6. “2030 Today, 2021,” Segment.com, 2021
7. “Getting Personal: Consumer Perspectives on AI in Marketing and Customer Service,” CDP.com, April 2022 
8. Gartner®, Gartner’s Customer Data Survey: The 360-Degree View of the Customer Is More Myth Than Reality, Benjamin Bloom, Lizzy Foo Kune, Mike McGuire, 30 November 2021
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, and MAGIC QUADRANT is a registered trademark of Gartner, Inc. and/or its affiliates and are used herein with permission. All rights reserved.
9. “Closing the data value gap,” Accenture, December 2021
10. “Reshaping retail banking for the next normal,” McKinsey & Company, 2020
11. “Bring Customers Into Clear Focus,” Gallup, January 6, 2021
12. “Responsible Marketing With First-Party Data,” BCG, May 18, 2020