GCP – Cloud Storage announcements at Next ‘23: a recap

Cloud Storage provides simple, scalable, secure, and cost-effective object storage for customers to store, access, and manage their data. This year at Google Cloud Next ‘23, we presented a number of new Cloud Storage capabilities designed to help you serve your storage needs for mission-critical applications of all kinds, including AI/ML and data analytics workloads. Here’s a concise recap of all the new features and capabilities that we announced.

Optimized storage for AI/ML and data analytics

Our customers increasingly rely on object storage when running their data-intensive workloads (AI/ML, batch analytics, streaming analytics, and high performance computing). To help you with this critical and fast-growing area, we announced multiple capabilities in the areas of programmability, performance, and manageability.

Programmability – APIs for easy integration into your business

Cloud Storage Fuse (generally available): Mount and access a cloud storage bucket as a local filesystem, optimized for AI/ML and GKE workloads (learn more)Pub/Sub to Cloud Storage subscriptions (GA): Simplify streaming data ingestion to your object storage buckets with just a few clicks (learn more)Transfer for HDFS (preview in Q4 ‘23): Use Storage Transfer Service to easily transfer petabytes of Hadoop/Spark workloads to Google Cloud

Performance – Optimal price/performance for your data-intensive workloads

Cloud Storage client library transfer manager (preview): Improved read/write performance in client libraries by parallelizing uploads and downloads (learn more)Anywhere Cache (preview in Q4 ‘23): Elastically scalable zonal SSD read cache to minimize your egress bandwidth costs at predictable low latenciesgRPC API (preview in Q4 ‘23): New Cloud Storage API option that provides more efficient routing for analytics workloads, reducing overall execution timeImproved Hadoop connector (GA in Q4 ‘23): Improved write performance for Hadoop/Spark workloads on Cloud Storage via parallelization and disk buffering

Manageability – End-to-end data lifecycle tailored for ease of use and security

Event-driven transfer (GA): Listen to event notifications to trigger Storage Transfer Service (learn more)Managed folders (preview in Q4 ‘23): Prefix/folder level fine-grained access control to enable simplicity and security for data-intensive workloads

Built for mission-critical workloads

No matter what workload you leverage on object storage, areas such as security, governance, availability, scalability, and management are important. Here are the capabilities we announced for mission-critical workloads of all kinds.

Intelligent storage – Easily understand and manage your data at scale

Storage Insights datasets (preview in Q4 ‘23): Visualize and analyze your storage usage and trends within BigQuery. This supplements currently available Storage Insights inventory reports (learn more)Autoclassfor existing buckets (preview in Q4 ‘23): Automatically transition objects to different storage classes based on last access time, without needing to create a new bucket

Enterprise Security and Governance: Safeguard assets while meeting security, governance, regulation, and compliance requirements

Custom audit information (GA): Additional headers to give user, application, and job context to audit logs (learn more)Object retention lock (in preview): WORM-compliant immutable object storage by setting retain-until policies on individual objectsAssured Workloads(GA): Integrated and packaged security product controls to help meet compliance requirements in United States public sector with added support for ITAR & IL5 compliance regimes. Includes controls for data residency, data access and data sovereignty for data at rest, in use and in transit. (learn more)Custom organization policy (preview in Q4 ‘23): Custom guardrails authored and enforced by customers on GCP resources (bucket) to meet their organization governance requirements

Resilience and Scalability: Enable geo-redundancy while maintaining strict data residency controls, disaster recovery, and real-time monitoring

Soft delete (preview in Q4 ‘23): Quickly recover your data from accidental deletion during a specified retention periodReplication monitoring (preview in Q4 ‘23): Near real-time replication status is now provided for default and turbo replication in dual-region and multi-region bucketsAdditional dual-regions (GA): Flexible and performant geo-redundancy with new support for Canada and Australia regions (learn more)

To learn more about how to leverage Cloud Storage in your application, try out one of the new Google Cloud Jump Start Solution tutorials. For a list of all of Google Cloud Next ‘23 announcements, check out the Next ‘23 Wrap-Up.