AWS – AWS Glue now provides job queuing

Today, AWS adds job queuing for AWS Glue jobs. This new capability enables you to submit AWS Glue job runs without needing to manage account level quotas and limits.

AWS Glue job queuing monitors your account level quotas and limits. If quotas or limits are insufficient to start a Glue job run, AWS Glue will automatically queue the job and wait for limits to free up. Once limits become available, AWS Glue will retry the job run. Glue jobs will queue for limits like max concurrent job runs per account, max concurrent Data Processing Units (DPU), and resource unavailable due to IP address exhaustion in Amazon Virtual Private Cloud (Amazon VPC). AWS Glue job queuing can be enabled on your jobs via the AWS Management console or API/CLI.

AWS Glue job queuing is available in all AWS commercial regions where AWS Glue is generally available. To learn more, visit the AWS Glue product page, our documentation, or blog post.

Read More for the details.