GCP – How Hackensack Meridian Health de-risked network migration using VPC Flow Logs

Network administrators rely heavily on VPC Flow Logs for visibility into their network traffic. Last year, we updated VPC Flow Logs to offer expanded network traffic visibility, extending beyond subnets to include VLAN attachments and VPN tunnels. This enhancement provides comprehensive monitoring of network traffic across your on-premises and multi-cloud environments.

Now, with VPC Flow Logs for VLAN attachments, you can export detailed telemetry data for your network traffic traversing Cloud Interconnect. This data encompasses essential information such as source and destination IP addresses, ports, protocols, bytes/packets transferred, timestamps, and other relevant metadata. These logs are crucial for a variety of use-cases, including network traffic analysis, troubleshooting, capacity planning, and maintaining compliance and security. Then, you can use Flow Analyzer to quickly analyze your VPC Flow Logs to gain valuable insights into your network without writing complex SQL queries. 

Sounds great, but how do you use it? Hackensack Meridian Health (HMH) is a leading not-for-profit healthcare organization and the largest hospital system in New Jersey. As a network of hospitals, urgent care centers, and physician practices, system reliability is extremely important and a cornerstone value of HMH. In this blog post, we demonstrate how HMH leveraged VPC Flow Logs and Flow Analyzer to analyze their Cloud Interconnect traffic prior to migrating their Google Cloud network to a new architecture design.

Let’s jump in.

Using VPC Flow Logs to prepare for migration

Last year, HMH was getting ready to migrate their critical, large-scale network to a newer Google Cloud network design. Before a migration of this scale, they wanted to use sankey diagrams to get a clear understanding of their most important hybrid traffic patterns. This analysis was the only way to accurately identify — and proactively plan for — the biggest risks that could cause disruption during the cutover.

“Getting a clear picture of our interconnect traffic always felt like a black box. Enabling VPC Flow Logs and feeding it into Flow Analyzer finally gave us the ‘who-is-talking-to-what’ map we needed. Identifying those critical traffic flows before we changed any routes was key to de-risking the entire migration.” – Randall Brokaw, Cloud Engineering Manager, Hackensack Meridian Health

To collect the necessary data, HMH enabled VPC Flow Logs on all of their VLAN attachments, then leveraged Flow Analyzer to easily aggregate the ingress and egress data. The following query components were used for ingress analysis:

• Source 

• Filter: Gateway type = INTERCONNECT_ATTACHMENT

• Organize Flows By: Gateway location, Gateway VPC network

• Destination

• Organize Flows By: GCE Instance Project, Google service type

These selections filter VPC Flow Logs to ingress traffic over Cloud Interconnect VLAN attachments, and aggregate the source traffic volume by Google Cloud region and VPC network.

The destination data was grouped by Compute Engine instance project to easily identify the destination application, since each application is deployed into a dedicated service project. However, since not all traffic is sent to Compute Engine VMs, incorporating the Google service type enabled them to account for traffic destined for Google APIs and Google VPC hosted services.

In your environment, the best flow parameters and destination grouping to conduct this analysis will depend on how your organization deploys applications on Google Cloud. For example, you can group by any of the available fields collected by VPC Flow Logs metadata, such as IP address and port, VPC subnet, GKE cluster, and more.

HMH then transformed the VPC Flow Logs traffic volumes into sankey diagrams. This required formatting each traffic flow into multiple three-column rows of {source, destination, weight}. For this analysis, the weight was the traffic volume displayed in Flow Analyzer, and source,destination corresponded to each layer of the sankey visualization in the following order:

1. Data center to Google Cloud region

2. Google Cloud region to VPC network

3. VPC network to application

Selecting “View the query in Log Analytics” from the Flow Analyzer console allows the traffic flows to be easily exported to Google Sheets and combined correctly for the diagram. Then using Google Charts, HMH created the sankey diagram:

Using VPC Flow Logs, HMH network engineers pinpointed critical cutover moments in their plan, allowing them to de-risk the migration through proactive monitoring and preparedness. This preparation proved its value when a migration issue was detected in 3 minutes and resolved in just 5 — slashing a resolution process that previously could have taken hours. This readiness was fundamental to the migration’s success.

This implementation uses Flow Analyzer which requires VPC Flow Logs to be stored in Cloud Logging. Alternatively, you have the option to forward VPC Flow Logs straight to BigQuery, bypassing Cloud Logging. From there, you can utilize visualization services like Looker to construct personalized dashboards and gain valuable insights.

VPC Flow Logs and Flow Analyzer for the win

HMH used VPC Flow Logs and Flow Analyzer to facilitate their network migration. But, by providing granular visibility into your Cloud Interconnect traffic, VPC Flow Logs can enable many other use cases, such as for capacity planning, cost attribution, and more. Enable VPC Flow Logs on your VLAN attachments today and leverage Flow Analyzer for insights into your traffic flow patterns.

To learn more, check out the VPC Flow Logs documentation or get started with Flow Analyzer to analyze your logs at no additional cost.