AWS – AWS Lambda announces Provisioned Mode for Kafka event source mappings (ESMs)

AWS Lambda announces Provisioned Mode for event source mappings (ESMs) that subscribe to Apache Kafka event sources, a feature that allows you to optimize the throughput of your Kafka ESM by provisioning event polling resources that remain ready to handle sudden spikes in traffic. Provisioned Mode helps you build highly responsive and scalable event-driven Kafka applications with stringent performance requirements.

Customers building streaming data applications often use Kafka as an event source for Lambda functions, and use Lambda’s fully-managed MSK ESM or self-managed Kafka ESM, which automatically scale polling resources in response to events. However, for event-driven Kafka applications that need to handle unpredictable bursts of traffic, lack of control over the throughput of ESM can lead to delays in your users’ experience. Provisioned Mode for Kafka ESM allows you to fine-tune the throughput of the ESM by provisioning and auto-scaling between a minimum and maximum number of polling resources called event pollers, and is ideal for real-time applications with stringent performance requirements.

This feature is generally available in all AWS Commercial Regions where AWS Lambda is available, except Israel (Tel Aviv), Asia Pacific (Malaysia), and Canada West (Calgary).

You can activate Provisioned Mode for MSK ESM or self-managed Kafka ESM by configuring a minimum and maximum number of event pollers in the ESM API, AWS Console, AWS CLI, AWS SDK, AWS CloudFormation, and AWS SAM. You pay for the usage of event pollers, along a billing unit called Event Poller Unit (EPU). To learn more, read Lambda ESM documentation and AWS Lambda pricing.

Read More for the details.