We are excited to announce the launch of two new disaster recovery features for Firestore: point-in-time recovery (PITR) and scheduled backups. These features will provide data protection against human errors and disasters. 

This is in addition to Firestore’s automatic and redundant data replication, which allows the service to guarantee up to 99.999% SLA availability with replica failovers demonstrating 0 RPO and 0 RTO.

Point-in-time recovery provides protection against accidental deletion or writes by giving you the ability to version control and query data from the past seven days, and surgically write the needed data back into your database.

Scheduled backups lets you backup your entire database on a chosen frequency of either daily and/or weekly onto cold storage. These backups can be used to restore to a new database in the same project. 

Point-in-time recovery example walkthrough

Let’s say you would like to enable point-in-time recovery. Below you will see how to enable this feature with the Google Cloud console: 

To enable point-in-time recovery, navigate to the “Disaster recovery” page and select the “EDIT” icon.

Select the checkbox for “Enable point-in-time recovery” and click “SAVE”.

The “Disaster recovery” page now includes “Earliest version time” and “Retention period”.

That’s it! Point-in-time recovery is enabled and will begin storing data. Please refer to the documentation for more information. 

Scheduled backups example walkthrough

Now we will walk you through how to generate a backup schedule using gcloud commands for both daily and weekly, as well as how to validate that the backups were created. 

To prepare, you’ll want to:

Upgrade gcloud to the newest version

Run `gcloud config set project <project-id>` for the project in which you want to manage backups 

To create a daily backup schedule, use the “backups schedules create” command. Below is an example of daily backup schedule creation:

To create a weekly backup schedule, use the “backups schedules create” command. Below is an example of weekly backup schedule creation:

To validate the backup schedules have been created successfully, use the “backup schedules list” command:

Next steps

For more information on how to set up and configure either of these features, simply check out our documentation for point-in-time recovery and scheduled backups for more information.

Thanks to Minh Nguyen, Senior Product Manager Lead for Firestore, for his contributions to this blog post.

For the past few years, Wayfair has been migrating from its on-premises data centers to Google Cloud. As part of this transition, we explored Vertex AI as an end-to-end ML platform and adopted several of its components, including Vertex AI Pipelines, Trainingand Feature Store as our preferred solution for new ML solutions going forward. Building on top of that work, our team decided to migrate our existing use cases to Vertex AI to fully benefit from the Google Cloud ecosystem and solve some issues that arose while using our legacy tooling. We detailed the engineering and MLOps aspects of that transition in this previous blog post. 

Below we share our perspective as data scientists describing our transition to Vertex AI. We’ll do that through an example — our delivery-time prediction use case.

Optimizing data collection with Vertex AI

Every data science solution starts with data. In the case of delivery-time prediction, we merge data from multiple sources: transactional data, delivery scans, information about suppliers and carriers which fulfill the orders and so forth. This routine is computationally expensive to execute every time we need training data, so we collect and aggregate the core data daily. Before migrating to Vertex AI, this data collection was handled with multiple pipelines overseen by different teams, which introduced unnecessary complexity and reduced development velocity. We moved and consolidated these distinct pipelines as a single serverless Vertex AI Pipeline that is developed and maintained exclusively by our data science team.

The first key benefit we experienced after moving to the new setup was having everything in a single repository, with unified versioning for everything inside, including all of our pipelines and the model code. The single repository tremendously improved traceability and debugging as we can now track changes at a glance. For example, we can easily check which model code was used in the training pipeline run, and we can then check the data collection pipeline to inspect the code that generated the data used for training. This is possible thanks to additional tooling that we built on top of Vertex AI and described in our previous blog.

Another change that made our lives easier is the serverless nature of Vertex AI. Because Google takes care of everything related to infrastructure and pipeline management, we reduced our dependency on the internal infrastructure teams that previously maintained a central Airflow server for our pipelines. Moreover, our pipelines run much faster and more reliably than before, as they get resources on demand and are not bottlenecked by other jobs running at the same time. 

Reduced dependency on other teams leads to the final key benefit of our Vertex AI transition: We now own the entire data collection workflow necessary for our model development. Vertex AI Pipelines make it easy to create or modify pipelines in just a few lines of code and they are well integrated with other Google products like BigQuery or Cloud Storage. What this means is that we, as data scientists, are now empowered to develop and productionize new features autonomously, drastically speeding up new model iterations. 

Offline experimentation at light speed

Now that dataflow is sorted, we can finally do some machine learning. Most of our ML happens in the development environment, where we use historical data for model development. You can see the typical experimentation pipeline below:

Supply chain is a dynamic creature. To test a hypothesis offline, we need to emulate model training and inference over a long period of time to reliably assess the model quality. The process requires a significant amount of parallelization. For one thing, we utilize parallel for-loops and concurrent pipeline runs extensively, exploiting the serverless nature of Vertex AI Pipelines. 

Next, parallelism helps us search for optimal hyperparameters. When splitting data into training, validation, and test sets, we first solve the hyperparameter optimization problem using a dedicated Vertex AI Pipelines component from the Google Cloud Pipeline Components library. The main job searches for the best combination of model hyperparameters by spawning workers that execute the training and validation script on different combinations. Apart from the speed gained, this component also offers a Bayesian optimization algorithm that improves the quality of the search. Vertex AI Pipelines and the Hyperparameter Tuning Job make our work much more efficient, and the setup allows us to iterate rapidly doing extensive offline experimentation as fast as possible. 

Once we’ve selected our hyperparameters, we run the final evaluation on the test dataset. We calculate a number of metrics to assess the model from multiple angles; this is the main output of an experiment. To store and navigate experiment results, we use Vertex AI Experiments whose simple interface allows us to log parameters and metrics from our pipelines. Then, we can either use the UI to analyze the results or pull those into a Jupyter Notebook, either locally or using Vertex AI Workbench.

Another convenience provided by Vertex AI is the seamless transition from a development environment to production. In our project, we retrain the live model weekly using the same components as described before. The only new part in the production pipeline is the model deployment step, where we make the model consumable by the live service. There, the model lifecycle is concluded. The process of model upgrade, as well as pipeline upgrade, is managed by a CI/CD system that you can learn more about in our last blog post. The tooling makes the release process streamlined and robust, allowing us to focus on data science problems and experimentation.

Build ML models faster

In summary, our Supply Chain Science team was able to iterate on model development more efficiently and autonomously by migrating to Vertex AI. The gains allow us to focus on improving Wayfair’s customer experience by running extensive offline experiments blazingly fast and easily releasing new ML models. In particular, delivery-time prediction really benefited from the new tooling — you can learn more about the model and its impact in our recent report.

It’s been a great experience partnering with the Vertex AI team and we recommend other data science groups try their platform. Among all the Vertex AI products, our team is particularly invested in Pipelines, Hyperparameter Tuning, and Experiments. Other teams at Wayfair have made use of even more capabilities provided by Vertex AI, so stay tuned to learn more about other use cases and how this state-of-the-art tooling powers data science.

Cloud Bigtable is a highly scalable, fully managed NoSQL database service that offers single-digit millisecond latency and an availability SLA up to 99.999%. It is a good choice for applications that require high throughput and low latency, such as real-time analytics, gaming, and telecommunications.

Cloud Bigtable change streams is a feature that allows you to track changes to your Bigtable data and easily access and integrate this data with other systems. With change streams, you can replicate changes from Bigtable to BigQuery for real-time analytics, trigger downstream application behavior using Pub/Sub (for event-based data pipelines), or capture database changes for multi-cloud scenarios and migrations to Bigtable.

Cloud Bigtable change streams is a powerful tool that can help you unlock new value from your data.

NBCUniversal’s streaming service Peacock uses Bigtable for identity management across their platform. The Bigtable change streams feature helped them simplify and optimize their data pipeline. 

“Bigtable change streams was simple to integrate into our existing data pipeline leveraging the dataflow beam connector to alert on changes for downstream processing. This update saved us significant time and processing in our data normalization objectives.” – Baihe Liu, Peacock

Actuating your data changes

Enabling a change stream on your table can easily be done through the Google Cloud console, or via the API, client libraries or declarative infrastructure tools like Terrafom.

Once enabled on a particular table, all data changes to the table will be captured and stored for up to seven days. This is useful for tracking changes to data over time, or for auditing purposes. The retention period can be customized to meet your specific needs. You can build custom processing pipelines using the Bigtable connector for Dataflow. This allows you to process data in Bigtable in a variety of ways, including batch processing, streaming processing, and machine learning. Or, you can have even more flexibility and control by integrating with the Bigtable API directly.

Cloud Bigtable change streams use cases 

Change streams can be leveraged for a variety of use cases and business-critical workloads. 

Analytics and ML
Collect event data and analyze it in real time. This can be used to track customer behavior to update feature store embeddings for personalization, monitor system performance in IoT services for fault detection or identify security threats, or monitor events to detect fraud.

In the context of BigQuery, change streams can be used to track changes to data over time, identify trends, and generate reports. There are two main ways to send change records to BigQuery: as a set of change logs or mirroring your data on BigQuery for large scale analytics.

Event-based applications 
Leverage change streams to trigger downstream processing of certain events, for example, in gaming, to keep track of player actions in real time. This can be used to update game state, provide feedback to players, or detect cheating.

Retail customers leverage change streams to monitor catalog changes like pricing or availability to trigger updates and alert customers.

Migration and multi-cloud
Capture Bigtable changes for multicloud or hybrid cloud scenarios. For example, leverage Bigtable HBase replication tooling and change streams to keep your data replicated across clouds or on-premises databases. This topology can also be leveraged for online migrations to Bigtable without disruption to serving activity.

Compliance
Compliance often refers to meeting the requirements of specific regulations, such as HIPAA or PCI DSS. Retaining the change log can help you to demonstrate compliance by providing a record of all changes that have been made to your data. This can be helpful in the event of an audit or if you need to investigate a security incident.

Learn more

Change streams is a powerful feature providing additional capability to actuate your data on Bigtable to meet your business requirements and optimize your data pipelines. To get started, check out our documentation for more details on Bigtable change streams, along with these additional resources:

Expanding your Bigtable architecture with change streams

Process a Bigtable change stream tutorial

Create a change stream-enabled table and capture changes quickstart

Bigtable change streams Code samples

About CNA

A key goal for insurers is to provide the best match between customer need, product offering and the premium charged. However, the price charged and eventual profitability is partially dependent on how well the insurer understands and assesses the risk being insured. This is usually a time-consuming and often subjective process that relies on the unique skills of the individual underwriter along with their ability to capture, analyze, and interpret vast amounts of associated data points. 

CNA is one of the largest commercial property and casualty insurance companies in the U.S. With over 120 years of experience, CNA provides a broad range of standard and specialized insurance products and services for businesses and professionals in the United States, Canada and Europe. Over the last three years, CNA has been building its data-analytics foundation on Google’s Data Cloud. This has been accomplished by consolidating hundreds of global data sources, leveraging Vertex AI automation to accelerate time-to-market for machine learning models, and delivering a series of executive reporting dashboards on key business performance metrics. 

Challenge: Underwriting flood risk for commercial properties

With the building of their analytics foundation complete, CNA has increasingly focused their attention on providing value-added analytic products at scale to their business units, specifically,  to enhance the underwriting experience by using advanced analytics technologies to deliver more robust insights within the underwriting ecosystem. This will enable underwriters to quickly and more accurately evaluate property risk and perform additional analysis to better understand the implications of various decisions and provide quality insurance products.

Effectively assessing underwriting flood risk was a priority area for CNA. This is a difficult process that must take into consideration a couple key factors: 

Flood hazards events are multidimensional in nature, with threats coming from coastal surges, fluvial (river), and pluvial (surface) sources. Coastal and fluvial risks are directly related to their proximity to water bodies. However, surface flooding is often spontaneous and typically occurs when intense rainfall or surface run-off overwhelms an urban drainage system. Pluvial flooding is often difficult to predict and urban areas are particularly vulnerable and have high levels of exposure to this hazard event. 

Flood hazards require a comprehensive understanding of the complex spatial relationships that exist between the commercial property being insured and its surrounding environment. The ability to identify any associated risk due to adjacent property location is also important. 

“We needed a way to improve how we assessed and understood flood risk. Shifting to a process that incorporates more robust floodplain data sources and applying geospatial analytics technologies to better understand the complex spatial relationships that exist is critical to providing underwriters with the accurate and timely insights that they need.” – Tom Stone, VP Aggregation and Catastrophe Management, CNA 

CNA’s current process for assessing flood risk had limitations across the following areas: 

Limited data coverage: The data used for analysis had gaps and only took into consideration fluvial and coastal water sources. Pluvial sources that accounted for two thirds of flood losses were not a part of the risk assessment process. 

Limited geographic extent: The process relied on a geocoded point location (property address) that did not capture the geographic extent of the entire site. The impact of the hazard on the entire site or even adjacent ones was challenging. 

Challenge with associated risk: Geocoded point locations could be anywhere on a property. These locations may intersect with existing floodplain data. However, in the absence of a precise intersection, the actual proximity of a property to a floodplain (close by, near to) could not be assessed. 

Solution: Powering flood risk assessment with BigQuery geospatial analytics

To solve this challenge CNA worked with Google Cloud and several third-party data vendors to develop a more sophisticated solution that addressed the current challenges with underwriting flood risk assessment

Floods and their associated impacts have a strong location component and geospatial analytics that uses location data (latitude, longitude) to better understand spatial relationships and patterns is the foundation of the solution. Improving the accuracy of geospatial analysis requires access to comprehensive data. CNA acquired several datasets on the natural and anthropogenic environment, including buildings, flood risk (coastal, fluvial and pluvial), land parcels, and more. 

These were large datasets that had both national and global geographic coverage. Because of this, CNA needed a data analytics solution that could store, process, and perform large-scale and computationally expensive geospatial analytics to better understand the complex relationships that exist between these datasets and the combined impact to both their current and future portfolio of business. 

“Over the years we have worked with Google Cloud to accelerate our data journey. BigQuery geospatial analytics was a natural fit for this flood risk assessment challenge that requires robust data management, powerful geospatial analytics functionality, and extensive computational scale to quickly analyze these large datasets.” – Dr. Pierre Braganza, Chief Enterprise Architect, CNA

Based on previous strategic investments, CNA was in a unique position to solve this challenge. They had already achieved data sufficiency at scalewith 90% of their data residing in a highly governed data warehouse built on BigQuery. BigQuery has the added benefit of possessing powerful capabilities for both geospatial data management and geospatial analytics. Using BigQuery for geospatial analytics lets you analyze geographic data using standard SQL geography functions. As these functions execute they harness the compute power of BigQuery, which makes this an ideal platform for rapidly analyzing the complex spatial relationships that exist within and between the datasets. 

Building an intelligent geospatial flood risk API 

The solution uses Dataflow and geobeam to land terabytes of geospatial data in raster and vector formats into BigQuery. Underpinning the solution is an intelligent API built on Google Kubernetes Engine that dynamically executes several BigQuery geospatial analytics functions to explore the spatial relationships between a potential underwriting location (latitude, longitude), the underlying associated parcel, the geographic extent of the related buildings, and their intersection with the floodplain. A multilevel flood risk score is automatically generated based on the locations’ spatial intersection (ST_Intersects) or proximity (ST_Buffer) to the floodplain zones, parcels and associated buildings.

This will be a significant improvement over the old process. 

“Previously when we geocoded a property address (latitude, longitude) it could be found anywhere on a property, which may not overlay with the floodplain. Now we have an intelligent API that assesses risk at multiple levels based on the geographic extent of the entire property.” – Tom Stone 

CNA can now understand flood hazards that affect any portion of a property as opposed to a single point location. This is of particular importance in instances where an insured location may be found on a site that has a geographic scope that crosses multiple floodplain zones. In such a scenario, a seemingly low-risk property could be determined to be more high risk based on this spatial relationship. 

The API is being integrated with CNA’s underwriting system and related business data. During the underwriting process, underwriters will automatically be provided with the various risk scores so more informed decisions can be made about the commercial property to be insured. The API will also allow CNA to analyze its entire business portfolio to better understand flood risk. 

Results: Improved risk decisions with geospatial technology 

CNA’s underwriting flood risk solution will drive improved insights for underwriters, which should provide numerous benefits: 

Underwriters are empowered with new functionality that quickly and accurately alerts them to locations on a property schedule that are susceptible to flooding while allowing for further analysis on potential implications. 

Through the solution, CNA will gain a more complete flood risk dataset that includes pluvial risk information on which to make more informed decisions. This should help reduce losses that are occurring based on inadequate pricing for this type of hazard. 

CNA will better understand the relationship between a location and the associated property, buildings and floodplain zones to which it belongs. This is critical for being able to classify both the direct and associated risks for a property that may span multiple floodplain zones. Preliminary analysis has shown an approximate 25% increase in the ability to provide a comprehensive view of flood risk. 

In the future, CNA plans to build additional geospatial visualization capabilities to allow for further exploration. The solution is designed to incorporate additional hazard types as business needs arise. CNA also has the ability to quickly perform a scenario-based analysis on an entire portfolio of business and its susceptibility to flood risk. 

By leveraging BigQuery for geospatial analytics, CNA tackled the spatial problem of being able to better understand and measure flood risk. With 90% of all data possessing a location component, geospatial analytics can be applied to other business areas and problem sets.

As a fully managed relational database service for MySQL, PostgresSQL and SQL Server, Cloud SQL lowers the cost of database ownership in several ways: it makes it easy to set up and operate a highly available database as well as scale with replicas, it provides security, it  automates database upgrades and maintenance, and it manages and monitors your database. It’s also easy to migrate to Cloud SQL using Google Cloud’s Database Migration Service. It’s no surprise then, that more than 95% of Google Cloud’s top 100 customers use Cloud SQL.

Google has now launched  Cloud SQL Enterprise Plus edition for MySQL and PostgreSQL. This new edition has all of the great benefits you get with Enterprise edition, but adds some important capabilities in availability and data protection: a 99.99% availability SLA that is inclusive of maintenance, near-zero-downtime upgrades, and up to 35-day point-in-time recovery capability. 

In addition to all these great new features, Enterprise Plus edition delivers several performance improvements including a data cache feature that allows you to seamlessly scale your workload. In this post we describe the performance results of Cloud SQL Enterprise plus edition on standard database benchmarks to showcase some of the improvements that you can gain when you choose it for your MySQL workloads.

New compute platform and architecture

Enterprise Plus edition gives MySQL users the option to enable a data cache feature, which can dramatically accelerate read performance. The data cache adds local SSD media to the Cloud SQL for MySQL instance, extending the MySQL buffer pool beyond memory to use local SSD. This allows read requests that cannot be serviced from data stored in the instance’s memory to be retrieved from local SSD rather than going out to instance storage, reducing latency and delivering 3x improvement in read throughput for MySQL workloads. The data cache is optional, but when you enable it, an optimal size of local SSD is provisioned on the instance depending upon the number of vCPUs you’ve configured.

Cloud SQL Enterprise Plus edition database instances leverage a different underlying hardware platform than Enterprise Edition, with a higher memory-to-vCPU ratio of 8 GB per vCPU. Even without any tuning, this platform delivers performance benefits when compared to Enterprise edition: the increased memory allows more data to be resident in memory when processing queries, therefore reducing latency associated with doing disk I/O operations during query processing. Cloud SQL Enterprise Plus edition also allows instances with up to 128 vCPU to allow your workloads to scale to higher levels.

With Enterprise Plus edition, we also improved Cloud SQL for MySQL’s write performance using several techniques and optimizations to reduce latency, resulting in up to 2X increased write throughput. This is due to a combination of more efficient log writes and a better CPU, resulting in reduced wait time for committing transactions, and increased throughput. This makes Enterprise Plus edition very well suited to applications with high data-ingestion rates. 

Combined with the read-performance improvements, these changes mean that Cloud SQL Enterprise Plus edition shines on mixed workload performance, as shown in the benchmark results below.

The blue bars represent the results when running on CloudSQL Enterprise Edition and the green bars are the results when running with Enterprise Plus with data cache enabled. Mixed workloads on Enterprise Plus edition show up to 171% improvement when compared to Enterprise edition.

Read-only workloads show an even more dramatic improvement when leveraging Enterprise Plus with data cache. The results of a read-only benchmark above show a 241% increase in queries per second (QPS) on a 32 vCPU instance and a 218% increase on a 64 vCPU instance. 

Go ahead, scale those MySQL applications

With the introduction of Cloud SQL for MySQL Enterprise Plus edition, you now have a great new option for scaling your applications. In this post, we covered the performance benefits using standard database benchmarks. The results demonstrate that the Cloud SQL Enterprise Plus edition offers significant performance improvements when compared to Enterprise edition, and also beats other major public cloud vendors’ comparative services’ MySQL performance. As a result, you can expect significant value in terms of price/performance when running your MySQL database applications with Google Cloud SQL for MySQL Enterprise Plus edition. 

Cloud SQL Enterprise Plus edition is now available in select Google Cloud regions.  To get started, provision an Enterprise Plus edition instance from the Google Cloud console or GCloud API.