Menu
Google AI Edge Portal: On-device machine learning testing at scale

GCP – Google AI Edge Portal: On-device machine learning testing at scale

,

Today, we’re excited to announce Google AI Edge Portal in private preview, Google Cloud’s new solution for testing and benchmarking on-device machine learning (ML) at scale. 

Machine learning on mobile devices enables amazing app experiences. But how will your model truly perform across the vast, diverse, and ever-changing landscape of mobile devices? Manually testing at scale – across hundreds of device types – is a laborious task that often requires a dedicated device lab. It’s slow, prohibitively expensive, and often out of reach to most developers, leaving you guessing about performance on users’ devices and risking delivering a subpar user experience.

Google AI Edge Portal solves the above challenges, enabling you to benchmark LiteRT models so you can find the best configuration for large-scale deployment of ML models across devices. Now, you can:  

  • Simplify & accelerate testing cycles across the diverse hardware landscape: Effortlessly assess model performance across hundreds of representative mobile devices in minutes.

  • Proactively assure model quality & identify issues early: Pinpoint hardware-specific performance variations or regressions (like on particular chipsets or memory-constrained devices) before deployment.

  • Lower device testing cost & access latest hardware: Test on diverse and continually growing fleet of physical devices (currently 100+ device models from various Android OEMs) without the expense and complexity of maintaining your own lab.

  • Unlock powerful, data-driven decisions & business intelligence: Google AI Edge Portal delivers rich performance data and comparisons, providing the crucial business intelligence needed to confidently guide model optimization and validate deployment readiness.

1 - Example benchmark result

Fig. 1. Interactive dashboard to gain insights on model performance across devices

In this post, we’ll share how our partners are already using Google AI Edge Portal, the user journey, and how you can get started. 

What our partners are saying

We’ve been fortunate to work with several innovative teams during the early development of Google AI Edge Portal. Here’s what a few of them had to say about its potential:

2 - Customer Story

How Google AI Edge Portal helps you benchmark your LiteRT models

  1. Upload & configure: Upload your model file via the UI or point to it in your Google Cloud Storage bucket.

  2. Select accelerators: Specify testing against CPU or GPU (with automatic CPU fallback). NPU support is planned for future releases.

  3. Select devices: Choose target devices from our diverse pool using filters (device tier, brand, chipset, RAM) or select curated lists with convenient shortcuts.

3 - Create New Benchmark Job

Fig. 2. Create a New Benchmark Job on 100+ Devices. (Note: GIF is accelerated and edited for brevity)

From there, submit your job and await completion. Once ready, explore the results in the Interactive Dashboard:

  • Compare configurations: Easily visualize how performance metrics (e.g., average latency, peak memory) differ when using different accelerators across all tested devices.

  • Analyze device impact: See how a specific model configuration performs across the range of selected devices. Use histograms and scatter plots to quickly identify performance variations tied to device characteristics.

  • Detailed metrics: Access a detailed, sortable table showing specific metrics (initialization time, inference latency, memory usage) for each individual device, alongside its hardware specifications.

4 - View Benchmark Result

Fig. 3. View Benchmark Results on the interactive Dashboard. (Note: GIF is accelerated and edited for brevity)

Help us shape the future of Google AI Edge Portal

Your feedback is crucial as we expand availability and enhance capabilities based on developer needs. In the future, we are keen to explore integrating features such as:

  • Bulk inference & evaluation: Run your models with custom datasets on diverse devices to validate functional correctness and enable qualitative GenAI evaluations.

  • LLM benchmarking: Introduce dedicated workflows and metrics specifically tailored for benchmarking the unique characteristics of large language models on edge devices.

  • Model optimization tools: Explore integrated tooling to potentially assist with tasks like model conversion and quantization within the portal.

  • Expanded platform & hardware support: Work towards supporting additional accelerators like NPUs, and other platforms beyond Android in the future.

aside_block
<ListValue: [StructValue([(‘title’, ‘$300 in free credit to try Google Cloud AI and ML’), (‘body’, <wagtail.rich_text.RichText object at 0x3e5bc475b670>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectPath=/vertex-ai/’), (‘image’, None)])]>

Join the Google AI Edge Portal private preview

Google AI Edge Portal is available starting today in private preview for allowlisted Google Cloud customers. During this private preview period, access is provided at no charge, subject to the preview terms.

This preview is ideal for developers and teams building mobile ML applications with LiteRT who need reliable benchmarking data across diverse Android hardware and are willing to provide feedback to help shape the product’s future. To request access, complete our sign-up form here to express interest. Access is granted via allowlisting. 

We are committed to making Google AI Edge Portal a valuable tool for the entire on-device ML community and we look forward to your feedback and collaboration!

Read More for the details.

Related Posts