Overview

In this blog, I want to focus on the autoscaling strategy for a common scenario that's both common and critical in modern cloud computing: scaling queue workers based on message queues.z

While some strategies might appear effective at first glance, they may not deliver the expected results in practice. We'll dissect these common approaches, spotlighting their pitfalls, before introducing a more robust method: queue-based scaling.

This technique not only excels in efficiency but also ingeniously leverages the principles of a negative feedback control system. I’ll discuss its characteristics and demonstrate why it stands out as the superior choice for autoscaling queue workers.

Understanding Queue Workers

Queue workers, integral to active applications, pose distinct challenges for effective autoscaling.

• Operational Nature: Queue workers function by continuously polling and processing tasks from a message queue. In an ideal setup, the presence of a sufficient volume of messages ensures that all consumer instances are fully utilized.

• Autoscaling Objective: The primary goal here is twofold – minimize the delay in message processing and maximize resource utilization.

  The balancing act is crucial: Over-provisioning can lead to inefficiency, as workers may be left idle without enough messages to process (a state known as 'starvation'). Conversely, under-provisioning can lead to increased message lag and processing delays, significantly impacting system performance.

<aside> 💡 Important: In this article we are only talking bout the case where lag is acceptable. If the goal is to process messages ASAP instead of maximize resource utilization. Over provision and starving of consumers are unavoidable.

</aside>

Traditional Autoscaling Strategies - An Overview

Strategy: CPU-Based Scaling

CPU-Based Scaling for active applications, watch the CPU usage of application.

metrics:
- type: Resource
  resource:
    name: cpu
    target:
      type: Utilization
      averageUtilization: 80

Key Characteristics

• Consumer CPU Usage: In a properly configured system, the CPU usage of consumers tends to reach 100% whenever there's a backlog (lag) in the message queue.
• Scaling Out: When message lag occurs, this strategy triggers the increase in the number of consumer instances, potentially up to the maximum limit with a short period as the lag may not clean up within a short period. This strategy will aggressively scale out the workload even if the lag is slightly.
• Scaling In: Reduction in the number of instances happens only when consumers start to starve, i.e., when they run out of messages to process.

Conclusion

While CPU-Based Scaling offers a straightforward approach, it’s obvious a bad choice for dynamic, active applications with queue-based workloads because