GCP – High throughput VM logging and metrics agent now in Preview

Running and troubleshooting production services requires deep visibility into your applications and infrastructure. Virtual machines running on Google Compute Engine (GCE) provide some system logs and metrics without any configuration required, but capturing application and advanced system data has required the installation of both a metrics agent and a logging agent. Today, we’re happy to announce the Preview of the new Ops Agent, which combines the metrics and logging agents into one, easy to install agent and supports a wider variety of use cases that require a higher throughput.

Higher throughput and improved resource efficiency

As customers have continued to build larger and more complex services on Google Cloud, we heard the feedback that you need a VM logging agent that can support a higher throughput (e.g. Nginx logs, Apache Access logs). This allows you to capture more data and avoid OutofMemory errors. In internal tests, the new Ops Agent has supported a greater than 1,000x higher throughput than our standard Cloud Logging agent, and this has been achieved with better efficiency for the underlying VM resources.    

The Ops Agent, currently in Preview, supports standard Linux and Windows operating systems and features simple YAML-based configurations for both logs and metrics, bringing greater consistency between logging and monitoring tools.

Automatic integration with Cloud Logging and Cloud Monitoring

As detailed in a previous post about the deep integration between Google Cloud infrastructure and our observability tools, just like our standard agents, the Ops Agent is pre-integrated with the observability tools available in the Google Cloud console. After you have installed the agent in your VM, application metrics and logs will automatically be routed to Cloud Logging and Cloud Monitoring without any action needed on your part. 

Installing the Ops Agent

Installing on multiple VMs

If you want to roll out the new Ops Agent to your fleet of VMs, we provide multiple options for installation via open source configuration management tools. In fact, we are the first public cloud to offer the ability to install the Ops Agent viaAnsible. We are also working to deliver support for Chef and Puppet in the first half of this year.

Managing fleets of VMs can also be done with our new Agent Policies, which let you declare the current and future VMs on which you want the agent installed. Use either the new  GCloud commands that have just been graduated to beta, or integrate directly with Terraform.

Installing on a single VM

If you want to test out the Ops Agent by installing on a single VM, you can do so using this installation guide. 

Get started today

We hope that you find the improved throughput, resource efficiency, and consolidation of the Ops Agent useful. If you have any questions or feedback during this Preview period, please reach out to us at google-cloud-ops-agent@google.com.