Explore AWS load balancer types, scaling mechanisms, and how traffic distribution and compute capacity work together.

AWS Load Balancing and Auto Scaling: How Traffic Distribution and Dynamic Capacity Work Together

An application starts receiving more traffic than its available compute capacity can handle. Response times increase, requests begin failing, and adding resources manually takes too long. AWS Load Balancing and Auto Scaling address two different parts of this problem: distributing incoming traffic and adjusting application capacity.

WHAT

AWS Elastic Load Balancing distributes incoming application traffic across registered targets, while AWS scaling services adjust compute or application capacity based on workload demand and configured policies.

Load Balancing

• Application Load Balancer (ALB): Operates at Layer 7 for HTTP and HTTPS traffic. Supports host-based and path-based routing, making it suitable for web applications and microservices.
• Network Load Balancer (NLB): Operates at Layer 4 for TCP, UDP, and TLS traffic. Designed for high-throughput, low-latency workloads and supports static IP addresses.
• Gateway Load Balancer (GWLB): Operates at Layer 3 and distributes traffic to virtual network appliances such as firewalls and intrusion detection systems.
• Classic Load Balancer (CLB): Previous-generation load balancer supporting basic Layer 4 and Layer 7 traffic distribution. Newer architectures generally use ALB or NLB.
• Target groups: Define the destinations that receive traffic. ALB supports instance, IP address, and Lambda function targets.

Auto Scaling

• EC2 Auto Scaling Groups: Maintain EC2 instance capacity using minimum, maximum, and desired capacity, along with launch templates.
• Scaling policies: Target tracking, step scaling, simple scaling, scheduled scaling, and predictive scaling adjust EC2 capacity according to workload requirements.
• ECS Service Auto Scaling: Adjusts the desired number of ECS tasks based on metrics and scaling policies.
• EKS workload scaling: Kubernetes mechanisms such as Horizontal Pod Autoscaler adjust pod counts. Cluster Autoscaler or Karpenter can help provide the required cluster compute capacity.
• Lambda scaling: Lambda automatically adjusts execution concurrency based on incoming events and requests. Provisioned concurrency can keep execution environments initialized to reduce cold-start latency.

WHERE

Load balancing and scaling are used across different application architectures, depending on how traffic enters the system and where application workloads run.

• Web applications: ALB distributes HTTP/HTTPS requests across EC2 instances or containerized application targets.
• Microservices: ALB routes requests to different target groups using host-based and path-based rules.
• High-throughput services: NLB distributes TCP, UDP, or TLS traffic across registered targets.
• Containerized applications: Load balancers distribute traffic to ECS or EKS workloads, while their respective scaling mechanisms adjust task or pod capacity.

Load balancer charges depend on the selected type and its pricing dimensions. Compute costs depend on the resources running the application, such as EC2 instances, ECS tasks, EKS worker nodes, or Lambda execution. Data transfer, monitoring, and supporting services can also contribute to the total workload cost.

WHY

Running an application with fixed capacity creates a limitation. During periods of high demand, available resources may not be sufficient to process incoming requests.

Load balancing distributes requests across multiple targets, reducing dependence on a single application instance. However, a load balancer does not create additional compute capacity when all targets are overloaded.

Scaling addresses this limitation by adjusting available resources based on workload demand. EC2 Auto Scaling can launch additional instances, ECS Service Auto Scaling can increase task counts, and Kubernetes scaling mechanisms can adjust pod and cluster capacity.

The two capabilities solve complementary problems:

• Load balancing distributes incoming traffic across registered targets.
• Scaling adjusts the capacity available to process that traffic.
• Health checks help prevent traffic from reaching unhealthy targets.
• Scaling policies determine when additional capacity is required.

The trade-off is between maintaining enough baseline capacity and paying for additional resources during peak demand. Scaling also takes time. New instances or containers must launch, initialize the application, and become ready before they can handle traffic.

The scaling mechanism must match the application platform. An EC2 Auto Scaling Group manages EC2 instances; it does not directly increase ECS task counts or EKS pod replicas. Those platforms use their own scaling mechanisms.

Validate the complete workload, including load balancer usage, compute capacity, expected traffic, and data transfer, using the AWS Pricing Calculator.

WHEN

Use Load Balancing and Auto Scaling when:

• Application traffic varies throughout the day or experiences unpredictable spikes.
• Multiple application targets need to receive incoming traffic.
• Compute or application capacity must increase or decrease based on workload demand.
• Applications need to distribute traffic across multiple targets or Availability Zones.
• The workload needs to maintain service availability while replacing or adding application resources.

The choice of load balancer depends on the traffic protocol and routing requirements. The scaling mechanism depends on the platform running the application: EC2, ECS, EKS, or Lambda.

HOW

Scenario: An e-commerce application experiences increased traffic during a seasonal sale.

A basic design places multiple EC2 instances behind an Application Load Balancer. The ALB distributes incoming requests, but the application still has a fixed number of instances. When traffic exceeds their combined capacity, response times can increase and requests may fail.

The improved design connects the ALB to an EC2 Auto Scaling Group. A target tracking policy monitors average CPU utilization or another suitable workload metric. When demand increases, Auto Scaling launches additional instances. Once they initialize, register with the target group, and pass health checks, they become eligible to receive traffic. When demand decreases, Auto Scaling gradually reduces capacity.

The same traffic-distribution principle applies to other platforms, but their scaling mechanisms differ. For example, an ALB can route requests to ECS tasks while ECS Service Auto Scaling adjusts task counts. EKS uses Kubernetes scaling mechanisms, while Lambda manages its own execution concurrency.

Scale-out:
Traffic / CPU increases
|
v
Scaling policy triggers
|
v
Auto Scaling launches EC2 instances
|
v
Instances initialize and register
|
v
Health checks pass
|
v
ALB sends traffic to new targets

Scale-in:
Demand decreases
|
v
Scaling policy triggers
|
v
Auto Scaling selects instances for termination
|
v
Connection draining and instance termination

Failure path:
Unhealthy target -> ALB stops routing traffic to target
Unhealthy instance -> Auto Scaling replaces it when configured

Monitoring:
Request count, target response time, healthy host count,
unhealthy host count, CPU utilization, and ASG capacity

1. Receive traffic: Users send requests to the Application Load Balancer through its configured listener.
2. Route requests: Listener rules determine the appropriate target group, and the ALB distributes requests across healthy registered targets.
3. Monitor demand: CloudWatch metrics provide workload signals used by the Auto Scaling policy.
4. Trigger scale-out: When the scaling policy's conditions are met, the Auto Scaling Group launches additional EC2 instances within its configured capacity limits.
5. Register targets: New instances initialize the application, register with the target group, and become eligible to receive traffic after passing health checks.
6. Trigger scale-in: When demand decreases, Auto Scaling reduces capacity according to the scaling policy and configured termination behavior.

The architecture introduces load balancer and compute costs as traffic and capacity change. These costs support traffic distribution and dynamic capacity, while minimum capacity determines the baseline cost of keeping the application ready.

TAKEAWAY

Load balancing distributes application traffic, while the appropriate scaling mechanism adjusts capacity for the platform running the application.