Automatic Pod Scaling

Introduction: When Fixed Replica Counts Aren't Enough

In previous lessons, you've been setting a fixed number of replicas in your deployments — maybe 2 or 3 pods to handle your application's traffic. But what happens when traffic suddenly doubles at 3 AM because your product goes viral on social media? Your fixed replica count means users experience slow response times or timeouts while your servers struggle under the load. By the time someone wakes up, checks monitoring dashboards, and manually runs kubectl scale to add more replicas, you've already lost customers and revenue. Conversely, during quiet periods, those extra replicas you added sit idle, wasting cluster resources and money. In this lesson, you'll learn how Horizontal Pod Autoscaling (HPA) solves this problem by automatically adjusting replica counts based on actual resource usage, ensuring your application scales up during traffic spikes and scales down during quiet periods without any manual intervention.

What is Horizontal Pod Autoscaling?

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

The Critical Prerequisite: Resource Requests

Here's something crucial that often trips up newcomers: HPA cannot work without resource requests defined in your Deployment. Remember in Lesson 2 when you learned about resource requests and limits? You learned that requests help Kubernetes schedule pods on nodes with enough capacity. Now you're about to discover another critical reason to always set resource requests: HPA uses them as the baseline for calculating utilization percentages. When you tell HPA to maintain 70% CPU utilization, it needs to know "70% of what?" The answer is: 70% of the CPU request you defined for each container.

There's also a second prerequisite: the metrics API must be available in your cluster. HPA retrieves current resource utilization data from the metrics API, which is typically provided by the metrics-server component. Without metrics-server running and healthy, HPA has no way to know how much CPU or memory your pods are currently using — it can only see the requests you defined, not the actual consumption. Most managed Kubernetes services (like EKS, GKE, or AKS) include metrics-server by default, but if you're running a local cluster with Kind or setting up your own cluster, you may need to install it separately. When metrics-server isn't available, kubectl get hpa will show <unknown> for current utilization values, and HPA won't make any scaling decisions.

Let's make this concrete with an example. If your container has a CPU request of 100m (100 millicores, or 0.1 CPU cores), and HPA is configured to maintain 70% utilization, HPA will try to keep each pod using around 70m of CPU on average. If your pods start using 90m each (90% utilization), HPA sees that you're exceeding the target and adds more replicas to spread the load. If usage drops to 40m per pod (40% utilization), HPA removes replicas because you're using less than the target threshold. Without a CPU request defined, HPA has no reference point — it can't calculate a percentage without knowing what 100% means. The HPA will show "unknown" for current metrics and won't make any scaling decisions.

This connects directly back to Lesson 2, but with a new perspective. Previously, you learned that resource requests ensure pods get scheduled on appropriate nodes and that containers get their fair share of resources. Now you understand that requests also serve as metrics anchors for autoscaling. This is why production Kubernetes deployments almost always include resource requests — they're not optional if you want reliable scheduling and autoscaling. If you forget to set requests and try to create an HPA, you'll see error messages or "unknown" values when you check kubectl get hpa, and your pods won't scale no matter how much load they experience.

Building a Deployment Ready for Autoscaling

Before you can create an HPA, you need a Deployment that's properly configured with resource requests. Let's build a deployment for a simple nginx web application that includes everything HPA needs to function. Create a new file called deployment-scalable.yaml and start with the basic deployment structure:

YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-scalable
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-scalable
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app

This looks familiar from previous lessons — we're creating a Deployment named web-app-scalable that will manage pods with the label app: web-app. We're starting with replicas: 2 as a baseline. This is important because it gives HPA room to scale in both directions: it can add replicas when load increases and remove replicas when load decreases. Starting with just one replica would mean HPA can only scale up, and starting with too many replicas might waste resources initially. Two replicas are a sensible default that provides high availability while leaving room for HPA to make adjustments.

Now let's add the pod template that defines the container configuration:

YAML

  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: nginx
          image: nginx:1.25
          ports:
            - containerPort: 80

  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: nginx
          image: nginx:1.25
          ports:
            - containerPort: 80

So far, this is a standard pod template: we're running the nginx:1.25 image, labeling the pod with app: web-app so the Deployment can manage it, and exposing port 80 for HTTP traffic. The critical part comes next — adding resource requests:

YAML

          resources:
            requests:
              cpu: "100m"

          resources:
            requests:
              cpu: "100m"

The resources.requests.cpu field specifies that each container needs 100m millicores of CPU. This is the field that HPA will use to calculate utilization percentages. When HPA sees that a pod is using 70 millicores, it knows that's 70% of the 100m request. When usage climbs to 90m, that's 90% utilization — above the target threshold, triggering a scale-up. The value 100m is deliberately modest because nginx is a lightweight web server that doesn't require much CPU under normal load. This low request value also makes it easier to trigger scaling during testing — you don't need to generate massive amounts of traffic to push CPU usage above 70%.

Here's the complete deployment-scalable.yaml file with all sections combined:

YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-scalable
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: nginx
          image: nginx:1.25
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: "100m"

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app-scalable
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: nginx
          image: nginx:1.25
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: "100m"

Save this file — you'll apply it in a few minutes. Notice that we're not setting resource limits in this example. While limits are valuable for preventing pods from consuming too many resources (as you learned in Lesson 2), they're not required for HPA to function. HPA only cares about the requests field because that's what it uses for utilization calculations.

Demo vs. Production Note: This example omits resource limits to keep the configuration minimal for learning. Production deployments typically set both requests and limits to prevent resource overconsumption, even though HPA only requires requests for its calculations. Always define both in real workloads.

Configuring the HPA Resource

Now that you have a Deployment with resource requests, you can create an HPA resource to control its scaling behavior. Create a new file called hpa.yaml. Let's build this configuration step by step, starting with the basic resource definition:

YAML

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa

The apiVersion is autoscaling/v2, which is the current stable version of the HPA API that supports multiple metric types, including CPU, memory, and custom metrics. The kind is HorizontalPodAutoscaler, which tells Kubernetes you're creating an HPA resource. We're naming it web-hpa to clearly indicate its purpose. Note that the HPA is a separate resource from the Deployment — you could delete and recreate the HPA without affecting the running pods, though autoscaling would stop working until you recreate it.

The most important section of the HPA configuration is the scaleTargetRef, which tells HPA which resource to control:

YAML

spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app-scalable

spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app-scalable

The scaleTargetRef section creates the connection between your HPA and the Deployment. The apiVersion: apps/v1 and kind: Deployment fields specify that you're targeting a Deployment resource. The name: web-app-scalable field must exactly match the name you used in deployment-scalable.yaml. This is how HPA knows which Deployment's replica count to adjust. You can only target one resource per HPA, but you can create multiple HPA resources if you need to autoscale multiple deployments.

Next, let's define the scaling boundaries:

YAML

  minReplicas: 2
  maxReplicas: 5

  minReplicas: 2
  maxReplicas: 5

The minReplicas and maxReplicas fields set the boundaries for autoscaling. HPA will never scale below minReplicas: 2, even if CPU usage is extremely low. This ensures you always have at least two pods for high availability — if one pod crashes, the other can continue serving traffic. HPA will never scale above maxReplicas: 5, even if CPU usage is extremely high. This prevents runaway scaling that could overwhelm your cluster or cause unexpected cloud costs. These boundaries give you control over the autoscaling behavior — you define the acceptable range, and HPA operates within those limits.

Finally, let's configure the metrics that HPA should observe:

YAML

  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

The metrics section tells HPA what to measure and what target to maintain. The type: Resource indicates we're using a standard Kubernetes resource metric (CPU or memory). The resource.name: cpu specifies that we want to monitor CPU usage. The target.type: Utilization means we're targeting a percentage of the requested resources rather than an absolute value. The target.averageUtilization: 70 is the key value — it tells HPA to maintain an average CPU utilization of 70% across all pods.

Why 70% instead of, say, 90% or 50%? This is a balancing act. If you set the target too high (like 90%), your pods are running near capacity most of the time, leaving little headroom for sudden traffic spikes. If you set it too low (like 30%), you're wasting resources by running more pods than necessary. A target around 60–80% is common in production because it keeps resources efficiently utilized while maintaining a buffer for unexpected load increases. The value 70% means HPA tries to keep average CPU usage at 70m per pod (70% of the 100m request you defined).

Here's the complete hpa.yaml file:

YAML

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app-scalable
  minReplicas: 2
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app-scalable
  minReplicas: 2
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Save this file — you're about to apply both configurations to your cluster. Understanding each section of this configuration is crucial because these settings directly control how your application responds to changing load. The scaleTargetRef determines what gets scaled, the minReplicas and maxReplicas set the boundaries, and the metrics section defines the scaling trigger. Together, these settings create a self-adjusting application that automatically finds the right number of replicas for the current load.

Detailed HPA Inspection and Monitoring

For detailed information about the HPA's current state and decision-making process:

Shell

kubectl describe hpa web-hpa

kubectl describe hpa web-hpa

The output provides extensive configuration and status information:

text

Name:                                                  web-hpa
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Mon, 15 Jan 2024 10:45:22 -0800
Reference:                                             Deployment/web-app-scalable
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  0% (0) / 70%
Min replicas:                                          2
Max replicas:                                          5
Deployment pods:                                       2 current / 2 desired
Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from cpu resource utilization
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:           <none>

Name:                                                  web-hpa
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Mon, 15 Jan 2024 10:45:22 -0800
Reference:                                             Deployment/web-app-scalable
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  0% (0) / 70%
Min replicas:                                          2
Max replicas:                                          5
Deployment pods:                                       2 current / 2 desired
Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from cpu resource utilization
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:           <none>

The Metrics section shows: resource cpu on pods (as a percentage of request): 0% (0) / 70%. The 0% (0) means current utilization is 0% with an absolute value of 0 millicores, compared to the target of 70%.

The Conditions section shows the HPA's operational state using three condition types:

AbleToScale: True — HPA is ready to make scaling decisions
ScalingActive: True — Metrics are available and HPA is functioning normally
ScalingLimited: False — Current replica count is within min/max range and not being constrained

These conditions help you troubleshoot when autoscaling isn't working as expected.

To watch the HPA status update in real-time:

Shell

kubectl get hpa web-hpa -w

kubectl get hpa web-hpa -w

This continuously displays HPA status and updates whenever values change:

text

NAME      REFERENCE                     TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
web-hpa   Deployment/web-app-scalable   0%/70%    2         5         2          2m

NAME      REFERENCE                     TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
web-hpa   Deployment/web-app-scalable   0%/70%    2         5         2          2m

The output refreshes whenever TARGETS or REPLICAS values change. Press Ctrl+C to stop watching.

To see the complete picture, open a second terminal and watch the pods simultaneously:

Shell

kubectl get pods -l app=web-app -w

kubectl get pods -l app=web-app -w

This shows pods being created or deleted as HPA makes scaling decisions:

text

NAME                                 READY   STATUS    RESTARTS   AGE
web-app-scalable-7d8f9b7c5d-k2m5p   1/1     Running   0          5m
web-app-scalable-7d8f9b7c5d-t9x3j   1/1     Running   0          5m

NAME                                 READY   STATUS    RESTARTS   AGE
web-app-scalable-7d8f9b7c5d-k2m5p   1/1     Running   0          5m
web-app-scalable-7d8f9b7c5d-t9x3j   1/1     Running   0          5m

When HPA scales up, new pods appear with STATUS: ContainerCreating that transition to Running. When HPA scales down, pods show STATUS: Terminating before disappearing. Watching both the HPA and pods simultaneously gives you a complete view of the autoscaling process. In the practice exercises, you'll generate load and watch HPA automatically add replicas to handle it, then scale back down when the load stops.

Introduction: When Fixed Replica Counts Aren't Enough

What is Horizontal Pod Autoscaling?

Join the 1M+ learners on CodeSignal

The Critical Prerequisite: Resource Requests

Building a Deployment Ready for Autoscaling

Configuring the HPA Resource

Applying the Deployment and HPA

Understanding HPA Status Output

Detailed HPA Inspection and Monitoring

Summary: Self-Adjusting Applications