Tibor Kiss

Software as a Service (SaaS) is the delivery method of choice for software vendors looking to serve a turnkey and reliable product experience to their end customers. There are many considerations a company must take into account when building a SaaS offering, one of which of course is the framework you will use to run your SaaS application. Since modern software development uses software containers, a natural and most popular choice for running modern SaaS platforms is Kubernetes, the popular container orchestrator. In this post we will go over the fundamentals of deciding what architecture to choose when building a SaaS platform on Google Kubernetes Engine (GKE).

Benefits of using GKE for SaaS applications

GKE is a managed, production-ready environment for deploying containerized applications. It is based on Kubernetes, an open-source system for automating deployment, scaling, and management of containerized applications, and was donated to the CNCF by Google, who is still the primary contributor to the project.

GKE offers a number of benefits for SaaS applications, including:

Globally available IP addresses that can be set up to route to one or more clusters depending upon the location of the inbound request. This enables advanced DR and application routing configurations. (to learn more about this, read about multi-cluster ingress and Google Cloud’s Global Load Balancer).Cost optimization: GKE provides cost optimization insights to help you align your infrastructure spend with your usage.Scalability: GKE can easily scale your applications up or down to meet demand. Current scale limits are 15,000 nodes per cluster, which leads the industry.Advanced storage options help you access data reliably, securely, and with high performance.

4 popular SaaS GKE architectures

When choosing a SaaS architecture, you must first think about your isolation requirements and the nature of your SaaS application. There is a trade off between cost and degrees of isolation, at the level of the Kubernetes namespace, node, and cluster. The cost will increase with each. In the following section, we outline architectures based on each in more detail, including their pros and cons. In addition to all the methods mentioned below, you can increase security on the host system by using GKE sandboxes. You can visit the main GKE Security overview page that also includes network security considerations here.

1. Flat multi-tenant application

One way to host a SaaS application is to set up single-ingress routing to a Kubernetes namespace with a copy of the SaaS application. The ingress router would have intelligence to serve data unique to the authenticated user. This setup is common for SaaS applications that don’t need to isolate users beyond the application’s software layer. This design is often only possible for applications that control tenancy via the software layer of the main SaaS application. CPU, memory, and disk/storage are scaled as the application grows with the default autoscaler without concern for which user is driving the most usage. Storage can be connected via persistent volume claims that are unique to each pod.

Pros:

Cluster and nodes are managed as a single and uniform resource

Cons:

The same underlying server is used for multiple tenants, and CPU spikes or networking events caused by one tenant (“noisy neighbors”) may affect other tenants.The same cluster control plane is used for multiple tenants, which means that any upgrades to the cluster apply to all tenants at the same time.User data is only isolated on the application layer, meaning problems in the application could expose one user’s data to another user’s.

2. Namespace-based isolation in a multi-tenant cluster

In this pattern, you set up single-ingress routing using the host path to route to an appropriate namespace, where there’s a copy of the application that’s dedicated to a given customer. This setup is common for customers who require highly-efficient resource isolation for their customers. Each namespace can be given a CPU and memory allocation and share excess capacity during spikes. Storage can be connected via persistent volume claims unique to each pod.

Pros:

Tenants can share resources in an isolated environment to boost efficiency and increase security.Cluster and nodes are managed as a single and uniform resource

Cons:

The same underlying server is used for multiple tenants, where CPU spikes or networking events caused by one tenant may affect others.The same cluster control plane is used for multiple tenants, which means that any cluster upgrades apply to all tenants at the same time.

3. Node-based isolation

Like above, here you set up single ingress routing using the host path to route to appropriate namespace which contains a copy of the application that’s dedicated to a given tenant. However, the containers running the application are pinned to specific nodes using labels. This provides the application with node-level isolation in addition to namespace isolation. This type of deployment is used when applications are very resource-intensive.

Pros:

Tenants have dedicated resources in an isolated environment.The cluster and nodes are managed as a single and uniform resource.

Cons:

Each tenant gets their own node and will consume infrastructure resources regardless of whether the tenant is using the application.The same cluster control plane is used for multiple tenants, which means that any upgrades to the cluster apply to all tenants at the same time.

4. Cluster-based isolation

The final pattern is to use a single unique ingress route to access each cluster, in which a specific version of the application lives that is dedicated to a given customer. This type of deployment is used when applications are very resource-intensive and also require the strictest security standards.

Pros:

Tenants have dedicated resources in completely isolated environments with a dedicated cluster control plane.

Cons:

Each tenant gets their own cluster and will consume infrastructure resources regardless of whether they are using the application.Clusters have to be updated independently, which can add a substantial operational burden.

Storage considerations

Once you’ve selected your baseline architecture, runtime storage is the next piece you’ll need to add to your setup. There are many different options for container-based storage for your application, and the best one will depend on your application’s specific requirements. Latency and persistence is perhaps the most important thing to consider in this decision. You can use the table below to help guide you on the different available options.

Local SSD (ephemeral storage):

Minimum capacity: 375 GB per disk
Use local SSDs if your applications:

Download and process data, such as AI or machine learning, analytics, batch processing, local caching, and in-memory databasesHave specialized storage needs and need raw block access on high-performance local storage.

Alternately, you may want to run specialized data applications and operate local SSDs like a node-level cache for your Pods. You can use this approach to drive better performance for in-memory database applications such as Aerospike or Redis (more info).

Persistent storage disk:
Minimum capacity: 10 GB
Use Persistent Disk storage if your clusters require access to high-performance, highly available durable block storage. A Persistent Disk volume is typically attached to a single Pod. This storage option supports the ReadWriteOnce access mode. GKE provides support for configuring Persistent Disk volumes with a range of latency and performance options. These disks are best used for general workloads or databases that require performant persistent block storage. Additionally they can be set up with regional redundancy (more info).

Object storage:
You can also connect Cloud Storage buckets to your applications. You can even mount storage buckets to your applications to interact with the remote files locally using the FUSE CSI driver (more info). This is a great option if you have pipelines that hydrate your storage outside of your core application.

Filestore (NFS):
Minimum capacity: 1 TB
Filestore provides centralized NFS storage that you can use to connect multiple pods to for read/write operations. This is a good option when multiple services in your cluster need read/write access to a central storage facility. (more info)

First-party services:
Finally, many GKE users connect their SaaS applications to hosted services such as CloudSpanner, Cloud SQL, or Redis to their clusters. This is a great option if you don’t need self-managed data services and can take advantage of Google Cloud managed services to lower your operational burden.

Customer spotlight

“Building Weaviate Cloud Services(WCS), on Google GKE enables us to scale our AI-native vector database elastically to meet our customers’ needs.”
– Etienne Dilocker, Co-Founder & CTO, Weaviate

“By building EDB BigAnimal™on Google GKE, we were able to leverage EDB’s existing Kubernetes expertise to deliver an intelligent managed database platform. GKE lets us embed automation in the platform with our operators while supporting a wide range of compute and storage options to our customers.”
– Benjamin Anderson, SVP of Technology and Strategy, EnterpriseDB

“Our next-generation serverless offering is tailored to meet our customers’ varied needs across their use cases for Observability, Security and Search, with zero administration overhead,” said Steve Kearns, Vice President, Product Management, Elastic. “For Google Cloud, we’ve chosen GKE, which enables us to utilize the latest compute and storage options in Google Cloud and achieve the most optimal combination between cost, performance and ease of management.“
– Steve Kearns, Vice President, Product Management, Elastic.

Learn more

If your organization is considering delivering its SaaS applications on GKE, you have a lot to think about: the benefits you hope to gain from doing so, the pros and cons of the various architectures, and your myriad storage options. You may also want to think about familiarizing yourself with GKE, GKE security, and how to distribute and monetize your SaaS app from within the Google Cloud. For more, check out the following resources:

Google Kubernetes EngineGKE security overviewGKE SandboxBuild Apps for the GCP Marketplace

Wouldn’t it be wonderful if you were always in the know about how your mission-critical workload was doing on Google Cloud? Perhaps you are a compliance officer that is in charge of regulatory compliance of your applications or a cloud administrator that cares deeply about application observability, to ensure that they run smoothly in the cloud.

At Google Cloud,we built Regional Persistent Disk with mission-critical workloads in mind — to provide high-availability by using synchronous replication of writes (RPO = 0) across two Google Cloud availability zones. As we continue to innovate on high availability, we are excited to introduce two new capabilities: the Regional Persistent Disk Replication State Dashboard, and the Replica State Cloud Monitoring metric, which can help you monitor and gain insight into your Regional Persistent Disk’s replication state.

Benefits of monitoring Regional Persistent Disk replication states

Monitoring for high-availability compliance audits: Regional Persistent Disks are used extensively as primary storage for mission-critical workloads (such as MySQL, SQL Server, and ElasticSearch) with strict high-availability compliance goals tied to them. Compliance audits are conducted regularly to verify that the workload and its underlying infrastructure meet the standards of availability and resilience as required by these goals. Without the ability to periodically monitor the current and historical replication state of Regional Persistent Disks, Google Cloud users have found it challenging to accurately prove that their application and workloads are compliant with their goals.Monitoring for high-availability compliance maintainability: In addition to periodically monitoring replication state for compliance reporting, Google Cloud users also want to continuously maintain high-availability and replication standards for their application and its data stored in Regional Persistent Disks. Maintaining replication standards can be challenging, as users must constantly inspect replication state to ensure that standards are being met. A proactive alerting mechanism that triggers when the replication state changes to a state that impacts compliance would greatly streamline the process of compliance maintainability.

Let’s take a deep dive into how Regional Persistent Disks enable these benefits for your mission critical workloads.

Replication States: A quick guide

Before diving deeper into how Regional Persistent Disk enables these benefits, it is useful to take a quick pit stop to understand the different replication states and why they are important to you. Regional Persistent Disks synchronously replicates data to two replicas in different Google Cloud zones in the same region. Depending on the state of the individual replicas, your regional Persistent Disk volume can be in one of the following replication states:

Fully replicated: Replicas in both zones are available, and are fully replicating at RPO=0. In this state, users experience no data loss if a zonal issue requires their virtual machine to be failed over to the other replicated zone. For organizations with a strict high-availability compliance goal, this is the best replication state to be in.Degraded: One of the replicas is offline and data is not being replicated between the two replicas. In this state, availability of the disk will likely be affected if a zonal error occurs on the remaining replica. The disk will typically not be in this state for very long as the Persistent Disk platform will be actively self healing to get back to a fully replicated state as soon as possible. To avoid potential exposure to data unavailability due to a failure of the remaining replica, it is best to enable snapshots or Persistent Disk Asynchronous Replication.Catching up: Regional Persistent Disks go from being in degraded state, to catching up and finally fully replicated if the replication can be self healed. This state is a useful precursor state to inform you that the disk is most likely working towards being fully replicated.

For more details on replication states, please see the replication state public documentation.

Monitoring for compliance audits

The current and historical Regional Persistent Disk replication status can be observed from the Regional Persistent Disk Replication State Dashboard, located in the Google Cloud console for all attached Regional Persistent Disks:

With this dashboard, you can see the Replication state of both replicas of a Regional Persistent Disk. A value of 1 indicates that a replica is fully synced with the replica in the other zone while a value of 0 indicates replication is not in sync with the other replica. Both replicas need to be in sync for the Regional Persistent Disk to be fully replicated. This dashboard allows you to easily and quickly view the current and historical replication status of a Regional Persistent Disk for high availability compliance audits and reports.

For more complex auditing, dive deeper into a more detailed view of each replica’s state by creating your own custom monitoring view using Cloud Monitoring’s Metric Explorer and the Cloud Monitoring metric called the “Regional Disk Replica State”. This metric records replication state information of each replica in 60-day time windows to allow you to gain insights such as:

How long has my Regional Persistent Disk been in a degraded replication state? Consequently the metric can be used to inform users on how long they have been compliant with their high availability compliance goals.Which replica is Out of Sync, causing my Regional Persistent Disk to be in a degraded state? This information is useful for troubleshooting and remediating replication issues with the disk.

For more information about this metric, see the Google Cloud Regional Disk Replica State documentation.

Since the Regional Disk Replica state is a Cloud Monitoring metric, this metric can be incorporated into many other monitoring and observability capabilities offered by Cloud Monitoring for more extensive audits:

Creating dashboards with other official Cloud Monitoring metrics to create comprehensive views in combination with other product metrics like compute and networking.Customize your audits with more flexible metric analysis using the Cloud Monitoring Metrics Explorer.Export the metric to external monitoring tools like Grafana and Prometheus to integrate Regional Persistent Disk replication status into your organization’s observability tools.

Monitoring for compliance maintainability — proactive alerting

An additional benefit of the Regional Disk Replica State metric is the ability to be integrated into Cloud Monitoring Alerting Policies. This allows for flexible and proactive alerting into a diverse set of alerting channels including SMS, Slack, and PagerDuty to integrate into your organizations infrastructure alerting needs.

You can set up alerts for changes in Regional Persistent Disk replication status, such as when a disk is in an unreplicated state, or when individual replicas switch state, making it easy to track and maintain compliance of high availability goals. For more details on enabling an alerting policy with the Regional Disk Replica State metric, see the Cloud Monitoring Alert Policy documentation.

Use it today

If you’re interested in leveraging the Regional Persistent Disk Replication State Dashboard and the Regional Disk Replica State Metric to gain insight into your Regional Persistent Disk high availability compliance, the dashboard and metric is available today for all projects with attached Regional Persistent Disks. For more information on how to get started, please visit the Regional Disk Replication Monitoring documentation page.

Editor’s note: In today’s guest post we hear from two digital events agencies, Gramercy Tech, a New York-based digital agency specializing in digital experiences,and Vatom, a Los Angeles-based company focusing on Web3 software, on their experience working with Google’s Cloud SQL to build and scale a virtual world.

Regardless of where your organization resides in relation to technology, now is not the time to stand still. But in times of uncertainty, it’s easy to do so. In our world of events and digital experiences, we are in an age of transformation, rethinking everything about how we serve attendees. To inspire us on this journey, we’ve leaned into the idea of building beyond: Rather than thinking about building digital experiences, we lean into the beyond — what does it mean to truly build beyond what was previously thought to be possible?

Recently, Google Cloud helped us answer this question while we were preparing to host a global, digital event. As event organizers, we are always looking for ways to make digital events more engaging and immersive for attendees, especially in the post-pandemic era. To do so, we created The Beyondverse, which combines Gramercy Tech’s Eventinity platform plus Vatom’s Web3 capabilities to help create a virtual conference where attendees can listen, learn, network, and participate all from their own homes, in a seamless fashion.

Eventfinity is a cloud-based platform that allows event organizers to create and manage virtual events of all sizes. With Eventfinity, organizers can create custom registration pages, manage speaker and exhibitor information, and deliver live and on-demand content. Eventfinity’s technology stack leverages Google Cloud’s serverless technologies and multi-regional deployments to efficiently scale Cloud Run across the globe. The Beyondverse, meanwhile, is hosted on the Vatom Web3 enterprise platform. Inside The Beyondverse, attendees can interact with objects, engage in conversations with other attendees, attend virtual events, and explore virtual spaces that replicate the look and feel of real-world events. Vatom’s infrastructure is based on a collection of microservices fronted either by a public API or an app and is delivered Google Cloud CDN fronted with Cloud Armor.

Underlying all this is Cloud SQL, Google Cloud’s managed database service for MySQL, PostgreSQL and SQL Server. With Cloud SQL, we hosted our databases and microservices on a shared VPC, connecting over 15,000 employees and partners simultaneously worldwide in the Beyondverse, with extremely low latency, and zero downtime. Cloud SQL also allowed us to establish read-replication globally, seamlessly integrate APIs, and enable a secure and encrypted experience for attendees. We kept core functions in our platform speedy by using Redis and Memcache which we were able to deploy using Memorystore.

This was in marked contrast to our previous cloud-based database. By migrating to Cloud SQL, we’ve been able to scale rapidly to meet the demands of our growing business. It allows us to stay focused on our core business while ensuring compliance with essential requirements such as high availability, reliability, disaster recovery, and security. Building and maintaining the platform was a seamless and managed process with Cloud SQL.

This collaboration with Google Cloud is a testament to the power of immersive experiences and the growing demand for these experiences. We look forward to continuing innovating and creating new digital adventures with Google Cloud.

In addition to Gramercy Tech and Vatom, more than 95% of Google Cloud’s top 100 customers use Cloud SQL to run their businesses. For more information on how Google Cloud’s managed relational database service for MySQL, PostgreSQL, and SQL Server can help you, visit https://cloud.google.com/sql.

Google Cloud Next is heading to Las Vegas in 2024 from April 9th to 11th, and I’m excited to announce that registration opens today. With a new location, fresh programming, hundreds of partners, and the latest in AI, analytics, security, infrastructure, collaboration and more, Google Cloud Next ‘24 is set to be our biggest event yet.

Early registration at the Google Cloud Next site is now open at a discounted rate of $999 USD* (that’s 50% off the full ticket price of $1,999 USD), and includes one year of Innovators Plus (limited quantity), so there’s no better time to register. Last year’s event with over 20,000 people sold out quickly — check out our 2 min recap video.

There has never been a more exciting time to learn more about cloud technology, and staying up-to-date on the latest in AI innovation from Google experts, partners, and your peers is critical for anyone looking to keep pace in this rapidly-evolving space. Throughout the conference, you’ll have opportunities to connect directly with Google engineers and experts, who can help you solve problems and brainstorm innovative solutions 1:1.

At Cloud Next ’24, we have learning opportunities for everyone.

For developers, there will be hundreds of learning sessions and an Innovators Hive with “how-to” content and demos, in addition to hands-on training on gen AI and other cutting-edge cloud technologies.For leaders we will highlight dozens of gen AI use cases in every function and industry and share details on how to move quickly from proof-of-concept to production as well as expected outcomes.For startups, we have a curated experience that includes startup sessions, networking opportunities, and the bustling Startup Lounge, where you can learn about how Google Cloud can help supercharge your startup’s growth.For everyone, we will have a huge show floor featuring the latest and greatest from our Google Cloud partner ecosystem as well as a customer innovation showcase.

There will be dedicated sessions for all levels across all roles, including application developers, architects and IT professionals, data and database engineers, data scientists and data analysts, DevOps/SREs/ITOps/platform engineers, IT managers and business leaders, productivity/collaboration app makers, security professionals, startup leaders/founders – there’s truly something for everyone!

Register today at our dedicated Google Cloud Next ‘24 site.

*The $999 USD early bird price is valid through 11:59 PM PT on Wednesday, January 31, 2024 or until it’s sold out.