APIs are an integral part of modern services, and the data they exchange is often highly sensitive. Without proper authentication, authorization, and protection against data leakage, your organization and your end users will face an increased risk of cyberattacks.
The Open Worldwide Application Security Project (OWASP) develops and publishes community-led documentation and standards for critical areas of software security, including APIs. APIs are estimated to comprise over half of internet traffic today.
That number is likely to climb as AI adoption grows, because AI already relies heavily on APIs for building foundation models, streamlining integration of AI capabilities into applications, facilitating interoperability between models running on different platforms, and providing continuous access to the real-time data needed to train and improve AI models.
Given the already large and growing reliance on APIs, organizations should implement an API security strategy. OWASP’s guidance on top 10 API security threats provides a starting point. We have taken their list and added mitigation recommendations for each risk they’ve identified. Our new whitepaper,Mitigating OWASP Top 10 API Security Threats, provides more details on each threat and how Apigee, Google Cloud’s API management platform, can help manage API risk.
aside_block
<ListValue: [StructValue([(‘title’, ‘$300 in free credit to try Google Cloud security products’), (‘body’, <wagtail.rich_text.RichText object at 0x3e24539fe670>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectPath=/welcome’), (‘image’, <GAEImage: Google Cloud>)])]>
What you can do about the OWASP top 10 API security risks
For organizations who are just getting started with their API security program, OWASP’s list of top 10 API security risks provides a good starting point. It represents the most critical vulnerabilities that organizations should address to protect their API systems. These threats are broadly categorized into themes of authorization, authentication, resource management, security misconfiguration, and third-party risks.
Authorization flaws, including Broken Object Level Authorization (BOLA), Broken Object Property Level Authorization (BOPLA),and Broken Function Level Authorization (BFLA), are particularly concerning as they allow attackers to bypass access controls and manipulate data or functionalities.
BOLA occurs when an API fails to enforce proper access controls on individual data objects, enabling attackers to access or modify data without proper authorization. BOPLA, on the other hand, arises when access control measures are not effectively enforced on individual properties within a data object, allowing attackers to manipulate sensitive attributes. BFLA occurs when specific functions or operations within the API lack adequate access control mechanisms, enabling attackers to perform unauthorized actions.
Authentication weaknesses, such as broken authentication, can lead to impersonation and unauthorized access. Unrestricted resource consumption and unrestricted access to sensitive business flows can also disrupt operations and expose critical data that can be exploited by attackers.
Security misconfiguration and improper inventory management of APIs can create additional vulnerabilities that attackers can exploit. Finally, unsafe consumption of third-party APIs introduces external risks, as vulnerabilities in those APIs can compromise the security of the consuming API.
Addressing these threats requires a multi-layered approach, including robust access controls, secure authentication mechanisms, proper resource management, thorough security configurations, and careful integration of third-party APIs.
Mitigating security risks with Apigee and Advanced API Security
Apigee, Google Cloud’s API management platform, enables API platform teams to program and deploy secure API proxies that can protect your backend services from these kinds of attacks. The chart below highlights some specific capabilities in Apigee and Advanced API Security that can help you keep your APIs protected from OWASP’s Top 10 API Security risks.
OWASP Top 10 API Security Risks (2023)
Apigee and Advanced API Security mitigation capabilities
API proxy security configuration checks and alerting, to check and alert on security misconfigurations across proxies (and you can use our API to integrate proxy security score checks into your CI/CD pipeline)
Teams who want to take a layered approach to API and application security can use Apigee and Advanced API Security together with a Web Application Firewall (WAF) like Cloud Armor. Cloud Armor’s robust protection against DDoS attacks — including L3/L4 DDoS defense and DDoS thresholds — can help increase protection against unrestricted resource consumption and other security threats.
aside_block
<ListValue: [StructValue([(‘title’, ‘Hear monthly from our Cloud CISO in your inbox’), (‘body’, <wagtail.rich_text.RichText object at 0x3e24539fe850>), (‘btn_text’, ‘Subscribe today’), (‘href’, ‘https://go.chronicle.security/cloudciso-newsletter-signup?utm_source=cgc-blog&utm_medium=blog&utm_campaign=FY23-Cloud-CISO-Perspectives-newsletter-blog-embed-CTA&utm_content=-&utm_term=-‘), (‘image’, <GAEImage: Cloud CISO Perspectives new header July 2024 small>)])]>
Get started on API security with Apigee
To learn more about how Apigee can help mitigate the OWASP top 10 API security threats, read our free whitepaper. It explores each threat outlined above in more detail, including specific product capabilities that can help protect against each threat.
Amazon Route 53 Traffic Flow now offers an enhanced user interface for improved DNS traffic policy editing. Route 53 Traffic Flow is a network traffic management feature which simplifies the process of creating and maintaining DNS records in large and complex configurations by providing users with an interactive DNS policy management flow chart in their web browsers. With this release, you can more easily understand and change the way traffic is routed between users and endpoints using the new features of the visual editor.
Now, Traffic Flow introduces a clearer way to craft DNS routing policies for many endpoints and multiple routing methods by moving configurations into a new sidebar, providing an undo/redo button, and introducing a new text editor for changing JavaScript Object Notation (JSON) files right within your browser. The JSON editor also includes syntax highlighting and can be used in conjunction with a new ‘Dark Mode’ theme to show where the policy edits should be made.
The new Traffic Flow experience is available globally, except in AWS GovCloud and Amazon Web Services in China. Traffic Flow pricing information can be found here and these enhancements are offered at no additional cost. To learn more about how to use Traffic Flow, visit our documentation or see this blog post.
Amazon S3 reduces pricing for S3 object tagging by 35% in all AWS Regions to $0.0065 per 10,000 tags per month. Object tags are key-value pairs applied to S3 objects that can be created, updated, or deleted at any time during the lifetime of the object.
S3 object tags help you logically group data for a variety of reasons such as to apply IAM policies to provide fine-grained access, to specify tag-based filters to manage object lifecycle rules, and to selectively replicate data to another AWS Region. Additionally, in AWS Regions where S3 Metadata is available, you can easily capture and query custom metadata that is stored as object tags.
S3 object tags are available in all AWS Regions including the AWS China and AWS GovCloud (US) Regions. This new pricing takes effect automatically in the monthly billing cycle starting on March 1, 2025. To learn more about object tags, refer to the documentation. For more pricing details, visit the S3 pricing page.
Customers can use regional processing profiles for Amazon Nova understanding models (Amazon Nova Lite, Amazon Nova Micro, and Amazon Nova Pro) in the Europe (Milan) and Europe (Spain) regions.
Amazon Bedrock is a fully managed service that offers a choice of high-performing large language models (LLMs) and other FMs from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, as well as Amazon via a single API. Amazon Bedrock also provides a broad set of capabilities customers need to build generative AI applications with security, privacy, and responsible AI built in. These capabilities help you build tailored applications for multiple use cases across different industries, helping organizations unlock sustained growth from generative AI while ensuring customer trust and data governance.
AWS AppSync Events is a fully managed service that allows developers to create secure and performant WebSocket APIs. Starting today, developers can use their AppSync Events APIs to publish events directly over WebSocket connections, complementing the existing HTTP API publishing capability. This enhancement enables applications to both publish and subscribe to events using a single WebSocket connection, streamlining the implementation of real-time features.
The new WebSocket publishing capability simplifies the development of collaborative applications such as chat systems, multiplayer games, and shared document editing. Developers can now maintain a single connection for bi-directional communication, reducing complexity and improving performance by eliminating the need to manage separate connections for publishing and subscribing to events. This approach helps reduce latency in real-time interactive applications by removing the overhead of establishing new HTTP connections for each event publication.
This feature is now available in all AWS Regions where AWS AppSync is supported.
To get started, developers can use their favorite WebSocket client. For more information, view our new blog post and visit the AWS AppSync documentation for detailed implementation examples and best practices.
Many Google Cloud customers have deep investments in third-party ISV security solutions such as appliances to secure their networks and enforce consistent policies across multiple clouds. However, integrating these security solutions into the cloud application environment comes with its own set of challenges:
Network re-architecture: Integrating third-party appliances for traffic inspection often necessitates a network redesign to route application traffic through them. With the high rate of change in a cloud application environment, this process can be error-prone, add operational overhead, and slow down application deployment time.
High cost of operation: The inability to selectively route traffic to third-party appliances for inspection leads to overprovisioning and increased costs. Customers often invest in larger, more expensive appliances to handle all their traffic, regardless of applications’ security inspection needs.
Difficulty meeting compliance requirements: Meeting security and regulatory requirements for an application deployment can be complex and often requires customers to implement custom tooling.
Today, we’re pleased to announce Network Security Integration to address these challenges. Network Security Integration helps you integrate third-party network appliance or service deployments with your Google Cloud workloads while maintaining consistent policies across hybrid and multicloud environments — without changing your routing policies or network architecture. Network Security Integration also enables comprehensive workload traffic visibility, advanced network security, and application/network performance monitoring. It uses Generic Network Virtualization Encapsulation, a.k.a. Geneve tunneling, to securely deliver traffic to third-party inspection destinations without modifying the original packets.
Additionally, Network Security Integration helps accelerate application deployments and compliance with a producer/consumer model. This allows infrastructure operations teams to provide collector infrastructure as a service to application development teams, enabling dynamic consumption of infrastructure as a service. Support for the hierarchical firewall policy management helps enforce compliance without introducing delays.
Network Security Integration offers two primary modes:
Out-of-band integration (GA): Mirrors desired traffic to a separate destination for offline analysis
In-band integration (Preview): Directs specific traffic to a third-party security stack for inline inspection
aside_block
<ListValue: [StructValue([(‘title’, ‘$300 to try Google Cloud networking’), (‘body’, <wagtail.rich_text.RichText object at 0x3dff697ca7c0>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectpath=/products?#networking’), (‘image’, None)])]>
Network Security Integration out-of-band
Running out-of-band, Network Security Integration transparently mirrors packets destined to and from the workload to a destination collector group. Geneve helps ensure secure transmission to the destination.
Running Network Security Integration out-of-band lends itself to the following use cases:
Implementing advanced network security – Use advanced offline analysis to detect known attacks based on predetermined signature patterns, and also identify previously unknown attacks with anomaly-based detection. Granular filtering capabilities ensure that vulnerable workload traffic is mirrored for advanced inspection.
Improve application availability and performance – Diagnose and analyze what’s going on over the wire instead of relying only on application logs. Network traffic analysis tools leverage machine learning and analytics to inspect mirrored packet data, baselining the normal behavior of the network and then detecting anomalies that might indicate potential availability or performance issues.
Support regulatory and compliance requirements – Finance and other regulated industries are required to capture and retain specific types of network traffic for a predetermined period to meet stringent requirements for auditing and forensic investigations.
Network Security Integration in-band
With in-band integration, traffic ingressing or egressing a workload can be intercepted and redirected to a security stack where the traffic is inspected for threats and compliance with security policy. The bump-in-the-wire implementation of in-band interception lets you inspect traffic between the VPC or even between different application components within the same VPC. With this, you can now shrink your security domain to as small as a workload, to deploy true Zero Trust security in your environment.
Choose to run Network Security Integration in-band for the following scenarios:
Integrate natively with Cloud Next Generation Firewall (NGFW) and third-party firewall – Network Security Integration simplifies the deployment of Google Cloud NGFW and third-party security solutions. It allows you to deploy third-party security services for traffic that requires additional security controls, while using Cloud NGFW’s distributed firewall features for optimized inspection.
Insert your preferred network security solution into brownfield application environments – Network Security Integration in-band is an elegant solution for integrating third-party security appliances directly into your existing network infrastructure, without requiring any modifications to your current routing configuration. By implementing it in-band, you can introduce additional layers of security and protection to your application traffic, helping to ensure comprehensive safeguarding against potential network threats.
What our partners are saying
This is what major partners had to say about Google Cloud’s Network Security Integration.
Palo Alto Networks
“Our partnership with Google Cloud continues with strong momentum, and today marks another milestone. Palo Alto Networks is partnering with Google Cloud to deliver advanced inline security protection for cloud and AI applications, significantly enhancing customer usability with a new deployment option. By integrating Palo Alto Networks AI-Runtime Security and VM-Series Virtual Firewalls with Network Security Integration, customers can rapidly secure their Google Cloud environment and AI applications, applying granular security policies based on a zero trust architecture.” – Jaimin Patel, Senior Director of Product Management, Palo Alto Networks
Fortinet
“Fortinet is partnering with Google Cloud to provide AI-powered threat intelligence for applications and workloads in Google Cloud by natively integrating with FortiGate next-generation firewalls. With the integration of Fortinet and Network Security Integration, customers are able to implement consistent cloud security policies and ensure faster and more reliable security response for their cloud networks.”– Vincent Hwang, Vice President of Cloud Security, Fortinet
Check Point
“We are excited to partner with Google Cloud to offer advanced threat prevention and secure connectivity across their global infrastructure. By securing the hybrid mesh with Network Security Integration and Check Point CloudGuard, our customers can stay free from cyber threats while automating management tasks and accelerating deployments across all Google Cloud regions.”– Kit Chee, Vice President, Global Strategic Partnerships, Check Point Software Technologies
Corelight
“Integrating with Google Cloud’s Network Security Integration empowers our customers to seamlessly adjust to the fluctuating demands of cloud environments. This integration enables our shared customers to expand the Corelight Network Detection and Response (NDR) value in the cloud, allowing comprehensive network visibility and threat detection. By adopting a straightforward, policy-driven strategy, organizations can effectively secure their Google Cloud deployments regardless of their scaling trajectory, optimizing both security and operational efficiency.” – Todd Wingler, VP, Global Alliances and Channels, Corelight
Trellix
“Trellix Virtual Intrusion Prevention System (vIPS) is a next-generation intrusion detection and prevention system (IDPS) that discovers and blocks sophisticated malware threats across the network. It uses advanced detection and emulation techniques, moving beyond traditional pattern matching to defend against stealthy attacks with a high degree of accuracy. Trellix has partnered with Google Cloud to integrate the Trellix vIPS with Network Security Integration. With the new architecture, Trellix and Google Cloud can meet the security challenges of the customers in a much faster and more scalable way and streamline the security adoption for our joint customers.” – Manish Kumar, Senior Software Architect, Trellix
cPacket
“cPacket is thrilled to partner with Google Cloud on their Network Security Integration rollout. When combined with cPacket’s Cloud Suite, customers can leverage best-in-class packet replication capabilities to multiple tools, powerful always-on packet capture and network analytics, and advanced visualization capabilities by utilizing these new in-band and out-of-band solutions delivered by Google Cloud.” – Trey Moczygemba, Sr. Cloud Product Manager, cPacket
Netscout
“NETSCOUT delivers actionable intelligence in Observability and Cybersecurity through real-time deep packet inspection (DPI). With NETSCOUT and Network Security Integration, customers gain powerful insights from end-to-end, packet-level visibility into their Google Cloud workloads and hybrid or multi-cloud connected applications, ensuring both performance and security.” – Tom Bienkowski, Senior Director, Security Product Marketing
An integrated security ecosystem
At Google Cloud, we’re committed to delivering enhanced visibility and top-tier security for customers’ network traffic and their workloads. With Network Security Integration, you can continue to use your third-party security solutions in your cloud environment, with lower costs, tighter integration, increased compliance, and no routing configuration changes. To learn more, visit the documentation for Network Security Integration. For Network Security Integration in-band (preview), contact your Google representative for access. We also encourage you to explore Cloud Next Generation Firewall (NGFW), our cloud-native, fully-distributed stateful inspection firewall engine that secures your network at cloud scale, enforced at each workload.
Amazon QuickSight a fast, scalable, and fully managed Business Intelligence service that lets you easily create and publish interactive dashboards across your organization is now available in Spain Region. QuickSight dashboards can be authored on any modern web browser with no clients to install or manage; dashboards can be shared with 10s of 1000s of users without the need to provision or manage any infrastructure. QuickSight dashboards can also be seamlessly embedded into your applications, portals, and websites to provide rich, interactive analytics for end-users.
With this launch, QuickSight expands to 23 regions, including: US East (Ohio and N. Virginia), US West (Oregon), Europe (Spain, Stockholm, Paris, Frankfurt, Ireland, London, Milan and Zurich), Asia Pacific (Mumbai, Seoul, Singapore, Sydney, Beijing, Tokyo and Jakarta), Canada (Central), South America (São Paulo), Africa (Cape Town) and AWS GovCloud (US-East, US-West).
Amazon Relational Database Service (RDS) for MySQL announces Amazon RDS Extended Support minor version 5.7.44-RDS.20250213. We recommend that you upgrade to this version to fix known security vulnerabilities and bugs in prior versions of MySQL. Learn more about upgrading your database instances, including minor and major version upgrades, in the Amazon RDS User Guide.
Amazon RDS Extended Support provides you more time, up to three years, to upgrade to a new major version to help you meet your business requirements. During Extended Support, Amazon RDS will provide critical security and bug fixes for your MySQL databases on Aurora and RDS after the community ends support for a major version. You can run your MySQL databases on Amazon RDS with Extended Support for up to three years beyond a major version’s end of standard support date. Learn more about Extended Support in the Amazon RDS User Guide and the Pricing FAQs.
Amazon RDS for MySQL makes it simple to set up, operate, and scale MySQL deployments in the cloud. See Amazon RDS for MySQL Pricing for pricing details and regional availability. Create or update a fully managed Amazon RDS database in the Amazon RDS Management Console.
Today, AWS is expanding service reference information to include resources and condition keys, providing a more comprehensive view of service permissions. Service reference information streamlines automation of policy management workflows, helping you retrieve available actions across AWS services from machine-readable files. Whether you are a security administrator establishing guardrails for workloads or a developer ensuring appropriate access to applications, you can now more easily identify the available actions, resources, and condition keys for each AWS service.
You can automate the retrieval of service reference information, eliminating manual effort and ensuring your policies align with the latest service updates. You can also incorporate this service reference directly into your existing policy management tools and processes for a seamless integration. This feature is offered at no additional cost. To get started, refer to the documentation on programmatic service reference information.
Today, we are announcing the availability of AWS Backup support for Amazon FSx for OpenZFS in 13 additional AWS Regions. AWS Backup is a policy-based, fully managed and cost-effective solution that enables you to centralize and automate data protection of AWS services (spanning compute, storage, and databases) and third-party applications. With this launch, AWS Backup customers can help improve business continuity, disaster recovery, and compliance requirements by protecting Amazon FSx for OpenZFS backups in additional Regions.
AWS Backup support for Amazon FSx for OpenZFS is added in the following Regions: Africa (Cape Town), Asia Pacific (Hyderabad, Jakarta, Osaka), Europe (Milan, Paris, Spain, Zurich), Israel (Tel Aviv), Middle East (Bahrain, UAE), South America (São Paulo), and US West (N. California).
Today, we are announcing the availability of AWS Backup logically air-gapped vault support for Amazon FSx for Lustre, Amazon FSx for Windows File Server, and Amazon FSx for OpenZFS. Logically air-gapped vault is a type of AWS Backup vault that allows secure sharing of backups across accounts and organizations, supporting direct restore to reduce recovery time from a data loss event. A logically air-gapped vault stores immutable backup copies that are locked by default, and isolated with encryption using AWS owned keys.
You can now protect your Amazon FSx file system in logically air-gapped vaults in either the same account or across other accounts and Regions. This helps reduce the risk of downtime, ensure business continuity, and meet compliance and disaster recovery requirements.
You can get started using the AWS Backup console, API, or CLI. Target Amazon FSx backups to a logically air-gapped vault by specifying it as a copy destination in your backup plan. Share the vault for recovery or restore testing with other accounts using AWS Resource Access Manager (RAM). Once shared, you can initiate direct restore jobs from that account, eliminating the overhead of copying backups first.
AWS Backup support for the three Amazon FSx file systems is available in all the Regions where logically air-gapped vault and respective Amazon FSx file systems are supported. For more information, visit the AWS Backup product page, and documentation.
Today, Amazon Web Services (AWS) announces the availability of Amazon GuardDuty Malware Protection for Amazon S3 in AWS GovCloud (US) regions. This expansion of GuardDuty Malware Protection allows you to scan newly uploaded objects to Amazon S3 buckets for potential malware, viruses, and other suspicious uploads and take action to isolate them before they are ingested into downstream processes.
GuardDuty helps customers protect millions of Amazon S3 buckets and AWS accounts. GuardDuty Malware Protection for Amazon S3 is fully managed by AWS, alleviating the operational complexity and overhead that normally comes with managing a data-scanning pipeline, with compute infrastructure operated on your behalf. This feature also gives application owners more control over the security of their organization’s S3 buckets; they can enable GuardDuty Malware Protection for S3 even if core GuardDuty is not enabled in the account. Application owners are automatically notified of the scan results using Amazon EventBridge to build downstream workflows, such as isolation to a quarantine bucket, or define bucket policies using tags that prevent users or applications from accessing certain objects.
GuardDuty Malware Protection for Amazon S3 is available in all AWS Regions where GuardDuty is available, excluding China Regions.
AWS Amplify Hosting is excited to offer Skew Protection, a powerful feature that guarantees version consistency across your deployments. This feature ensures frontend requests are always routed to the correct server backend version—eliminating version skew and making deployments more reliable.
You can enable this feature at the branch level in the Amplify Console under App Settings → Branch Settings. There is no additional cost associated with this feature and it is available to all customers.
This feature is available in all 20 AWS Amplify Hosting regions: US East (Ohio), US East (N. Virginia), US West (N. California), US West (Oregon), Asia Pacific (Hong Kong), Asia Pacific (Tokyo), Asia Pacific (Osaka) Asia Pacific (Seoul), Asia Pacific (Mumbai), Asia Pacific (Singapore), Asia Pacific (Sydney), Canada (Central), Europe (Frankfurt), Europe (Stockholm), Europe (Milan), Europe (Ireland), Europe (London), Europe (Paris), Middle East (Bahrain) and South America (São Paulo).
AWS announces the general availability of one new larger sizes (48xlarge) on Amazon EC2 I8g instances in US East(N. Virginia) and US West(Oregon) regions. The new size expand the I8g portfolio supporting up to 192vCPUs, providing additional compute options to scale-up existing workloads or run larger sized applications that need additional CPU and memory. I8g instances are powered by AWS Graviton4 processors that deliver up to 60% better compute performance compared to previous generation I4g instances. I8g instances use the latest third generation AWS Nitro SSDs, local NVMe storage that deliver up to 65% better real-time storage performance per TB while offering up to 50% lower storage I/O latency and up to 60% lower storage I/O latency variability. These instances are built on the AWS Nitro System, which offloads CPU virtualization, storage, and networking functions to dedicated hardware and software enhancing the performance and security for your workloads.
I8g instances offer instance sizes up to 48xlarge, 1,536 GiB of memory, and 45 TB instance storage. They are ideal for real-time applications like relational databases, non-relational databases, streaming databases, search queries and data analytic.
Amazon S3 Tables now seamlessly integrate with Amazon SageMaker Lakehouse, making it easy to query and join S3 Tables with data in S3 data lakes, Amazon Redshift data warehouses, and third-party data sources. S3 Tables deliver the first cloud object store with built-in Apache Iceberg support. SageMaker Lakehouse is a unified, open, and secure data lakehouse that simplifies your analytics and artificial intelligence (AI). All data in SageMaker Lakehouse can be queried from SageMaker Unified Studio and engines such as Amazon EMR, AWS Glue, Amazon Redshift, Amazon Athena, and Apache Iceberg-compatible engines like Apache Spark or PyIceberg.
SageMaker Lakehouse provides the flexibility to access and query data in-place across S3 Tables, S3 buckets, and Redshift warehouses using the Apache Iceberg open standard. You can secure and centrally manage your data in the lakehouse by defining fine-grained permissions that are consistently applied across all analytics and ML tools and engines. You can access SageMaker Lakehouse from Amazon SageMaker Unified Studio, a single data and AI development environment that brings together functionality and tools from AWS analytics and AI/ML services.
The integrated experience to access S3 Tables with SageMaker Lakehouse is generally available in all AWS Regions where S3 Tables are available. To get started, enable S3 Tables integration with Amazon SageMaker Lakehouse, which allows AWS analytics services to automatically discover and access your S3 Tables data. To learn more about S3 Tables integration, visit the documentation and product page. To learn more about SageMaker Lakehouse, visit the documentation, product page, and read the launch blog.
AWS announces the general availability of Amazon SageMaker Unified Studio, a single data and AI development environment that brings together functionality and tools from AWS analytics and AI/ML services, including Amazon EMR, AWS Glue, Amazon Athena, Amazon Redshift, Amazon Bedrock, and Amazon SageMaker AI. This launch includes simplified permissions management that makes it easier to bring existing AWS resources to the unified studio. SageMaker Unified Studio allows you to find, access, and query data and AI assets across your organization, then collaborate in projects to securely build and share analytics and AI artifacts, including data, models, and generative AI applications. Unified access to your data is provided by Amazon SageMaker Lakehouse and governance capabilities are built in via Amazon SageMaker Catalog.
Amazon Q Developer is now generally available in SageMaker Unified Studio, providing generative AI-powered assistance across the development lifecycle. Amazon Q Developer streamlines development by offering natural language, conversational interfaces that simplify tasks like writing SQL queries, building ETL jobs, troubleshooting, and generating real-time code suggestions. The Free Tier of Amazon Q Developer is available by default in SageMaker Unified Studio; customers with existing Amazon Q Developer Pro Tier subscriptions can access additional features.
Selected capabilities from Amazon Bedrock are also generally available in SageMaker Unified Studio. You can rapidly prototype, customize, and share generative AI applications using high-performing foundation models and advanced features such as Amazon Bedrock Knowledge Bases, Amazon Bedrock Guardrails, Amazon Bedrock Agents, and Amazon Bedrock Flows to create tailored solutions aligned to your requirements and responsible AI guidelines.
See Supported Regions for a list of AWS Regions where SageMaker Unified Studio is generally available. To learn more about SageMaker Unified Studio and how it can accelerate data and AI development, see the Amazon SageMaker Unified Studio webpage or documentation. You can start using SageMaker Unified Studio today by selecting “Amazon SageMaker” in the AWS Console.
Amazon S3 Tables now offer table management APIs that are compatible with the Apache Iceberg REST Catalog standard, enabling any Iceberg-compatible application to easily create, update, list, and delete tables in an S3 table bucket.
These new table management APIs, that map directly to S3 Tables operations, make it easier for you to get started with S3 Tables if you have a custom catalog implementation, need only basic read and write access to tabular data in a single S3 table bucket, or use an APN partner-provided catalog. For unified data management across all of your tabular data, data governance, and fine-grained access controls, you can use S3 Tables with SageMaker Lakehouse.
The new table management APIs are available in all AWS Regions where S3 Tables are available, at no additional cost. To learn more about S3 Tables, visit the documentation and product page. To learn more about SageMaker Lakehouse, visit the product page.
At Definity, a leading Canadian P&C insurer with a history spanning over 150 years, we have a long tradition of innovating to help our customers and communities adapt and thrive. To stay ahead in our rapidly evolving industry, we knew a unified data foundation was key to realizing the business and customer experience opportunities offered by modern analytics and AI.
While our legacy on-premises Cloudera platform had served us well, it could no longer support our growing needs for scale, innovation, and harnessing the power of data and AI. So, we embarked on a critical mission: modernizing our data infrastructure.
Legacy limitations stifling innovation
We faced a combination of interconnected challenges, which impact many organizations today:
Limited scalability and AI/ML workload support: Our existing infrastructure, constantly running at 80% utilization, was stretched thin. Processing billions of daily events for real-time analytics and scaling AI and ML workflows was a constant battle, limiting our ability to gain timely insights and develop innovative, data-driven products and experiences.
Data silos, fragmented insights: Our data resided in various systems, creating a fragmented view of our business. This made it difficult to get a holistic understanding of our customers and hindered initiatives like building a comprehensive customer 360º view and delivering personalized recommendations at a moment of relevance.
Escalating costs: Maintaining and scaling our Cloudera platform, which hosted massive data volumes (200TB compressed, 1PB uncompressed), was increasingly expensive and diverting valuable fiscal and people resources away from strategic priorities.
Faced with these pressing issues, the timing of our next renewal presented a strategic window of opportunity. We had a critical decision to make — migrate both technology and business platforms within 10 months or invest in upgrading our legacy Cloudera environment.
aside_block
<ListValue: [StructValue([(‘title’, ‘$300 in free credit to try Google Cloud databases’), (‘body’, <wagtail.rich_text.RichText object at 0x3ea733fef130>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectPath=/products?#databases’), (‘image’, None)])]>
Building a unified data and AI platform with Google Cloud
We chose Google Cloud and its powerful duo, BigQuery and Vertex AI, to build the Strategic Data Platform (SDP) — our new modern data analytics platform. BigQuery’s serverless architecture, unmatched scalability, built-in ML capabilities, and seamless integration with Vertex AI made it the ideal solution to power our data-driven transformation.Our migration was a remarkably fast-paced effort, carried out in close collaboration with with Google Cloud and Quantiphi, a Google Cloud partner.
Like many enterprises, we adopted a hybrid approach. We retained Databricks on Google Cloud for specific ETL workloads, utilizing Quantiphi’s expertise in converting legacy systems. At the same time, we migrated the bulk of our data processing to BigQuery for optimal cost-efficiency and performance. We also used Cloud Composer to orchestrate our complex data pipelines and ensure secure, private connectivity within our Google Cloud environment, a crucial requirement for handling sensitive customer data.
As a result, our dedicated team of over 100 Definity employees completed the migration in just ten months — 50% faster than the industry average. This rapid transition was aided by innovative tools, such as the “nifi-migration” solution built by Google Cloud Consulting. This open-source tool provided a visual and highly configurable way to automate real-time data flow between different systems, minimizing disruption and helping us surpass our initial migration timeline expectations.
Our CTO, Tatjana Lalkovic, who championed this effort to consolidate our structured and multimodal data to accelerate our AI/ML use cases, shared her perspective on the impact of our decision, saying:
“As we reimagined where data and AI could take our business, industry, and customer experience, Google Cloud BigQuery and Vertex AI stood out as modern, enterprise-ready serverless solutions prepared to meet the AI moment — not just today but for the foreseen future. The speed and success of this migration has created a lot of trust in our partnership and has been a significant boost to our digital transformation to streamline operations, improve products, and better serve customers.”
Strategic Data Platform – High level design on Google Cloud
Transforming insurance with data and AI
The results of our migration to BigQuery and Vertex AI has been transformative for Definity. We’ve seen exceptional user satisfaction, with the SDP achieving a remarkable Net Promoter Score (NPS) of 9.9 out of 10. The move has also saved us millions on our annual spend on non-strategic technologies and delivered a roughly 75% reduction in planned downtime for our digital platforms.
Performance has also dramatically improved, with processes to gain insights that once took days now completing in an average of 4.5 hours. Moreover, migrating to Google Cloud has helped us increase agility and innovation. We’re now able to double our business releases per year — achieving a 30% increase in testing automation, a 63% improvement in deployment time, and a 10x faster infrastructure setup.
By combining structured and unstructured data in BigQuery, we’ve unlocked new analytical possibilities and improved price-performance. This unified data foundation has empowered our business intelligence tools with richer, more comprehensive data, leading to more informed business decisions. The seamless integration with Vertex AI has enabled us to develop, deploy, and scale AI models, driving innovation in areas like fraud detection, automated intake, and personalized call center experiences. At the same time, we benefit from Google Cloud’s strong commitment to data security and privacy, helping us to strengthen our security posture and keep our customers safe.
As our VP of Data, Ron Mills, said:
“BigQuery’s serverless architecture has been a game-changer. The ‘nothing to manage’ approach is a huge differentiator. For enterprises like us that are migrating from on-prem clusters constantly running at 80% capacity, it’s like night and day.”
Lessons learned from our migration journey
Migrating a core data platform is a significant undertaking, and we’ve learned a lot along the way. For other organizations considering the same journey, here are some key takeaways from our experience:
One team, one goal: Foster a collaborative environment where technology and business teams, vendors, contractors, and consultants work together seamlessly towards a shared objective.
Leadership trust and commitment: Executive leadership trust in the delivery team’s decision-making is crucial for maintaining momentum and navigating challenges.
Be bold: Don’t be afraid to think outside the box, make timely decisions, and be prepared to adapt quickly to unforeseen setbacks.
Plan for the unknown: Anticipate potential roadblocks and have a core team dedicated to developing alternative solutions and addressing unforeseen issues.
Strong business partnership: A trusted relationship with business teams is essential for smooth user acceptance testing, change management, and avoiding unnecessary disruptions during the migration.
Balanced governance: Independent governance should provide guidance and support calculated risk-taking, acting as a partner in problem-solving rather than a blocker.
Motivated team: Cultivate a team-oriented environment where ownership of the project extends beyond leadership to every team member.
Transparent communication: Maintain open and consistent communication among all stakeholders (in our case, over 250 people) to ensure everyone is aligned and informed.
Fast fail and incremental delivery: Avoid a “big bang” approach. Embrace incremental releases (we aimed for 2-5 daily releases) to learn quickly, adapt, and iterate.
Parallel run: Plan for a parallel run of your systems on both the legacy and target cloud platforms to ensure a smooth transition and validate the new environment.
A data-driven future with limitless potential
Our migration to BigQuery and Vertex AI is just the first step in Definity’s data transformation journey. With a modern, scalable, and AI-ready data foundation now in place, we are empowered to unlock even greater value from our data and continue to lead innovation in the insurance industry. We are excited about the possibilities that lie ahead and are already actively developing our next AI use cases, including several focused on legal summarization and IT functions. We are confident that our partnership with Google Cloud will be instrumental in helping us achieve our goals.
Get ready to dive deep and level up your cloud skills at Google Cloud Next ’25. Whether you’re a seasoned pro or just starting your cloud journey, you’ll have more learning opportunities at Next than ever before. From hands-on challenges to expert-led workshops on AI and ML, Next ’25 (April 9-11, 2025) is your chance to transform your knowledge into real-world expertise.
The first-ever Skills Challenge: your chance to win big
This year, we’re launching a new, on-the-ground game: The Skills Challenge. Think of it as your personal learning adventure at Google Cloud Next, complete with:
Hands-on labs: Master practical skills.
Certification kickstarters: Pave your way to certification.
AI Agent Builder Bar: Experience AI agent development.
Quizzathon at Makerspace: Test your Google Cloud knowledge and win.
Leaderboard competition: See how you stack up against your peers and compete for grand prizes.
Early access: Be the first to try new gamification features on Google Cloud Skills Boost.
Don’t miss out on limited-edition swag and bragging rights on the leaderboard displayed at the Learning and Certification booth. Top the charts by the third day of Next for a chance to win a grand prize.
aside_block
<ListValue: [StructValue([(‘title’, ‘Get hands-on experience for free’), (‘body’, <wagtail.rich_text.RichText object at 0x3ea731b74bb0>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectPath=/welcome/’), (‘image’, None)])]>
Dive into expert-led workshops
Developers, these lab-style sessions are for you. They’re led by knowledgeable experts who can help you build better, faster, and smarter applications. Secure your spot today by adding training sessions to your agenda, such as:
Gain insights from breakout sessions and lightning talks
From in-depth technical learning to strategic insights for leaders, explore topics like credentialing a modern workforce, and building a culture of continuous learning with expert-led presentations and panel discussions. Hear from customers about their technical use cases and gain valuable takeaways about their specific approaches and solutions.
Recharge at the Google Developer Experts and Certified Lounge
Connect with your peers, recharge, and refuel at an exclusive lounge reserved for Google Developer Experts. Enjoy light refreshments, meetups, and a photowall to capture your Next ’25 experience. When you register for Next, remember to identify yourself asGoogle Cloud Certifiedon yourProfile page for easy lounge entry!
Can’t wait? Start learning today with Google Cloud Skills Boost
Don’t wait until Next to start your learning journey. Jump-start your skills with courses, labs, learning paths and more on Google Cloud Skills Boost. Join theInnovators program to get 35 credits at no cost and use them to keep learning on Skills Boost.
We’ll see you April 9-11 at Google Cloud Next ‘25! Register today.
Today’s insurance customers expect more: simple digital services, instant access to service representatives when they want to discuss personal matters, and quick feedback on submitted invoices. Meeting these demands has become increasingly difficult for insurers due to rising inquiry volumes, a shortage of skilled workers, and the loss of expertise as employees retire.
Recognizing the growing need for immediate and accurate responses, SIGNAL IDUNA, a leading German full-service insurer, particularly prominent in health insurance, introduced a cutting-edge AI knowledge assistant, powered by Google Cloud generative AI.
“We’ve pioneered to unlock the power of human-AI collaboration: To redefine process efficiency by bringing together technology and subject matter experts to deliver exceptional customer experiences,” said Johannes Rath, board member for Customer, Service, and Transformation at SIGNAL IDUNA.
SIGNAL IDUNA, in collaboration with Google Cloud, BCG and Deloitte, has developed an AI knowledge assistant that empowers service agents to quickly and accurately resolve complex customer inquiries. This innovative solution uses Google Cloud AI, including Google’s multimodal Gemini models, to help agents find relevant documents and provide comprehensive answers 30% faster — ultimately, enhancing customer satisfaction.
aside_block
<ListValue: [StructValue([(‘title’, ‘$300 in free credit to try Google Cloud AI and ML’), (‘body’, <wagtail.rich_text.RichText object at 0x3ea734674910>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectPath=/vertex-ai/’), (‘image’, None)])]>
The Challenge: Meeting modern expectations
Like many organizations in the insurance sector, SIGNAL IDUNA faced significant operational burdens. The complexity of insurance products, along with the growing demand for immediate and accurate responses, often leads to bottlenecks that can impact service experiences.
For example, prior to introducing its AI knowledge assistant, service agents had to manually search thousands of internal documents for hundreds of different tariffs to find the information needed to answer questions or resolve customer issues — including, insurance conditions, tariff information, guidelines, and standard operating procedures. As a result, 27% of inquiries required further escalation to other departments or specialists, resulting in delayed resolutions, increased costs, and potential damage to reputation.
Though complex, SIGNAL IDUNA prioritized this process as one of its top gen AI use cases, developing an AI assistant to help agents provide quick and accurate answers to customer inquiries, particularly those about health insurance. The AI knowledge assistant is grounded in more than 2,000 internal documents for more than 600 different tariffs, allowing agents to ask questions in natural language and receive accurate answers, significantly reducing the time spent searching for relevant information.
A deep dive into SIGNAL IDUNA’s gen AI system
Working with Google Cloud, BCG, and Deloitte, SIGNAL IDUNA built a sophisticated generative AI architecture using Google Cloud’s AI platform, Vertex AI, and utilized Gemini 1.5 Pro’s long-context capabilities to develop an AI knowledge assistant that can provide quick and accurate access to the right information within a vast collection of documents. The system employs multiple steps to aggregate and process extensive information from diverse sources, ensuring agents can access the complete context necessary to effectively address customer inquiries.
Here’s a breakdown of the key steps:
An end-to-end architecture diagram
1. Data pre-processing and extraction The knowledge base is built from various document types, which are typically in PDF format, including policy documents, operating procedures, and general terms and conditions.
SIGNAL IDUNA utilizes a hybrid approach that combines Layout Parser in Google Cloud Document AI and PDFPlumber to parse these PDFs and extract the text content. While the Layout Parser is responsible for extracting the text segments, SIGNAL IDUNA enhances the extraction of tables with PDFPlumber if the quality of the PDFs allows. The extracted texts are then cleaned, chunked by Google’s Gecko multilingual embedding model, and enhanced with additional metadata, enabling the ability to process and analyze the information later effectively.
For storing the vectorized texts, Google Cloud SQL for PostgreSQL is used with the pgvector PostgreSQL extension, which provides a highly effective vector database solution for our needs. By storing vectorized text chunks in Cloud SQL, SIGNAL IDUNA benefits from its scalability, reliability, and seamless integration with other Google Cloud services, while pgvector empowers efficient similarity search functionality.
2. Query augmentation Query augmentation generates multiple queries to improve the formulation of user questions for both document retrieval from the vector store and answer generation. The original question is reformulated into several variants, creating three versions in total: the original query, a rewritten query, and an imitation query. These are used then to retrieve relevant documents and generate the final answer.
For the rewritten query, the system uses Gemini Pro 1.5 to correct spelling errors in the original question. Additionally, the query is expanded by adding synonyms for predefined terms and tagging specific terms (e.g., “remedies,” “assistive devices,” “wahlleistung/selective benefits”) with categories. The system also uses information about selected tariffs to enrich the query. For example, tariff attributes, such as brand or contract type, are extracted from a database and appended to the query in a structured format. These specific adjustments make it possible to handle special tariff codes and add further context based on tariff prefixes.
The imitation query uses Gemini Pro 1.5 to rephrase the question to mimic the language of technical insurance documents, improving the semantic similarity with the source material. It considers conversation history and handles age formatting.
3. Retrieval First, the system checks the query cache, which stores previously answered questions and their corresponding correct answers. If the question, or one very similar to it, has already been successfully resolved, the cached answer is retrieved, helping to provide a rapid answer. This efficient approach ensures quick access to information and avoids redundant processing.
The accuracy of the cache is maintained through a user feedback loop, which identifies correctly answered questions to be stored in the cache through upvotes. A downvote on a cached answer triggers an immediate cache invalidation, ensuring only relevant and helpful responses are served. This dynamic approach improves the efficiency and accuracy of the system over time. If no matching questions are found in the query cache, the retrieval process falls back on the vector store, ensuring that the system can answer novel questions.
After retrieving any relevant information chunks from the query cache or vector store, the system uses the Vertex AI ranking API to rerank them. This crucial process analyzes various signals to refine the results, prioritizing relevance and ensuring the most accurate and helpful information is presented.
Ensuring complete and accurate answers is paramount during retrieval, and SIGNAL IDUNA found that some queries required information beyond what was available in the source documents. To address this issue, the system uses keyword-based augmentations to supplement the final prompt, providing a more comprehensive context for generating responses.
4. Generation The answer generation process involves three key components: the user’s question with multiple queries, retrieved chunks of relevant information, and augmentations that add further context. These elements are combined to create the final response using a complex prompt template.
Delivering a near real-time experience is crucial for service agents, so SIGNAL IDUNA also streams the generated response. During development, minimizing latency based on the input posed a significant technical hurdle. To address this issue, SIGNAL IDUNA reduced processing times using asynchronous APIs to help stream data and handle multiple requests. Currently, the system has achieved an average response time of approximately 6 seconds, and SIGNAL IDUNA is experimenting with newer faster models to reduce this time even further.
5. Evaluation Rigorous evaluation is essential for optimizing Retrieval Augmented Generation (RAG) systems. SIGNAL IDUNA uses the Gen AI evaluation service in Vertex AI to automate the assessment of both response quality and the performance of all process components, such as retrieval. A comprehensive question set, created with input from SIGNAL IDUNA’s service agents, forms the basis of these automated tests.
Here’s a closer look at how Looker helps evaluate the AI knowledge assistant:
Chunk retrieval: First, SIGNAL IDUNA evaluates retrieval of relevant information chunks. Metrics at this stage help assess how effectively the model identifies and gathers the necessary information from the source data. This includes tracking gen AI metrics, such as recall, precision, and F1-score, to pinpoint areas for improvement in the retrieval process. This is crucial as retrieving the correct information is the foundation of a good generated response.
Document reranking: Once the relevant chunks are retrieved, they’re reranked to prioritize the most pertinent information. The Looker dashboard allows monitoring the effectiveness of this reranking process.
Generated vs. expected response comparison: The final stage involves comparing the generated response with the expected response. SIGNAL IDUNA evaluates the quality, accuracy, and completeness of the generated output, utilizing large language models (LLMs) to score the similarity between the generated response and the expected response.
Explanation generation: To understand the reasoning behind an LLM’s evaluation, SIGNAL IDUNA generates explanations for its judgments. This provides valuable insights into the strengths and weaknesses of the generated responses, helping the developers identify specific areas for improvement.
This multi-stage evaluation approach provides SIGNAL IDUNA a holistic view of the model’s performance, enabling data-driven optimization at each stage. The Looker dashboard plays a vital role in visualizing these metrics, making it easier for the developers to identify areas where the model excels and where it needs improvement.
Real-world impact: AI-powered efficiency and productivity
To determine whether the AI assistant provided measurable added value for its workforce, SIGNAL IDUNA conducted an experiment with a total of 20 employees (internal and with external providers). During the experiment, customer requests were processed with and without the AI knowledge assistant to assess its impact.
One of the key benefits observed was a reduction in processing time. Searching across numerous data sources used to be a time-consuming process. The experiment showed that using the AI knowledge assistant reduced the core processing time (information search and response formulation) by approximately 30% and increased the quality of the response based on expert evaluations. The time saved was particularly notable for employees with less than two years of experience in health insurance.
In addition, the AI knowledge assistant significantly increased the case closure rate. Health insurance is a very complex field, and the use of external service providers means that not every employee can always answer every customer question. With support from the AI knowledge assistant, SIGNAL IDUNA’s case closure rate increased by approximately 24 percentage points, rising from 73% to almost 98%.
Scaling for the Future
“Together with Google, we at SIGNAL IDUNA have successfully applied gen AI to one of our core business processes” Stefan Lemke, CIO at SIGNAL IDUNA, said. “Now, it’s time to scale this powerful technology across our entire organization. We’re not just scaling a tool, we’re scaling innovation, learning, and the possibilities of what we can achieve.”
Gen AI offers enormous potential for optimizing processes and developing innovative solutions. With its innovative approach — business teams experimenting with the technology in a decentralized manner and developing customized applications — SIGNAL IDUNA is primed to pioneer the next generation of insurance solutions and services.
At the same time, SIGNAL IDUNA is establishing central standards to scale insights gained across the company and tap into the combined power of its teams, resources, and lines of business. This strategic decision has helped create valuable resources like code libraries, infrastructure blueprints, and centrally offered services.
By combining agility with established standards and best practices, SIGNAL IDUNA can now react quickly to new requirements, setting a new standard for efficiency and customer satisfaction.
This project was delivered by the following core team members, Max Tschochohei, Anant Nawalgaria, and Corinna Ludwig by Google, and Christopher Masch, Michelle Mäding from SIGNAL IDUNA
