Autoscaling¶
Status: AI draft, 2026-11-16. Not yet verified by me. The HPA commands were run on a local kind cluster (Kubernetes 1.37) that has no metrics server, so they show the failure case; scaling behaviour is from the documentation.
In one line¶
Kubernetes scales on three levers: the HorizontalPodAutoscaler changes how many pods run, the VerticalPodAutoscaler changes how big each pod's requests are, and a node autoscaler (Cluster Autoscaler or Karpenter) adds nodes when pods cannot be scheduled. The HPA computes ceil(current replicas × current metric / target) every 15 seconds, and CPU targets are percentages of requests.
The plain idea¶
When load rises, you can run more copies of the app, give each copy more CPU and memory, or buy more machines to put them on. Kubernetes has one automatic controller for each:
flowchart TD
L["Load rises"] --> H["HPA: more pods<br/>(core Kubernetes)"]
L --> V["VPA: bigger requests per pod<br/>(add-on)"]
H --> P["New pods are Pending:<br/>no node has room"]
V --> P
P --> N["Node autoscaler: adds a node<br/>(Cluster Autoscaler or Karpenter)"]
N --> S["Pods scheduled and running"]
They chain: the pod autoscalers create demand, and the node autoscaler supplies capacity (Cluster Autoscaler vs Karpenter). Each one needs measurements to decide, and that is where most autoscaling problems start.
How it works¶
Where the numbers come from¶
| API | Serves | Provided by |
|---|---|---|
metrics.k8s.io (resource metrics) |
CPU and memory per pod and node | metrics-server, an add-on; stable v1 since Kubernetes 1.37 |
custom.metrics.k8s.io |
Metrics about Kubernetes objects, such as requests per second per pod | An adapter, for example Prometheus Adapter |
external.metrics.k8s.io |
Metrics from outside the cluster, such as queue length | An adapter or KEDA |
kubectl top reads the same resource metrics API. If kubectl top fails, a CPU-based HPA cannot work either.
The HPA loop¶
Every 15 seconds (the controller manager's default sync period) the HPA controller:
- Reads the target's pods through the Deployment's selector, and their metric values.
- Computes
desiredReplicas = ceil(currentReplicas × currentMetricValue / desiredMetricValue). - Skips the change if the ratio is within the tolerance (0.1 by default) of 1.0.
- Applies its
behaviorrules (below) and writes the new count to the Deployment'sscalesubresource. The Deployment then does the rest (Deployments and rollouts).
Utilisation is relative to requests. averageUtilization: 60 means 60% of each pod's CPU request, not of a core and not of the limit (Requests, limits and QoS). A pod with no CPU request cannot be measured as a percentage.
Pods that are not ready yet, or are missing metrics, are set aside and treated conservatively, so a slow-starting app does not trigger runaway scaling.
behavior: how fast it may move¶
The default rules, from the documentation:
behavior:
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 100
periodSeconds: 15
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Percent
value: 100
periodSeconds: 15
- type: Pods
value: 4
periodSeconds: 15
selectPolicy: Max
- Scale up at once: double, or add 4 pods, whichever is more, every 15 seconds.
- Scale down carefully: use the highest recommendation of the last 5 minutes, which stops flapping when load is spiky.
selectPolicy: Disabledturns off scaling in one direction, for example never scaling down automatically.
Scaling to zero¶
CPU and memory can only be measured on running pods, so a CPU-based HPA cannot scale to zero. Since Kubernetes 1.37, an HPA with an object or external metric (a queue length, for example) may set minReplicas: 0; the HPAScaleToZero feature is beta and on by default. While at zero, a Service has no endpoints and does not hold requests, so this suits queue workers, not request-driven web apps.
KEDA is an add-on built for event-driven scaling: a ScaledObject names a source (a queue, a stream, a Prometheus query, a cron schedule), and KEDA creates and drives an HPA for it, including scaling to and from zero.
VPA in brief¶
The VerticalPodAutoscaler is an add-on (a CRD and three components: recommender, updater, admission controller). It watches real usage and recommends requests. Its updateMode:
| Mode | What it does |
|---|---|
Off |
Only writes recommendations to .status. A safe way to right-size by hand. |
Initial |
Applies requests only when pods are created. |
Recreate |
Evicts pods whose requests are far off, so they come back resized. |
InPlaceOrRecreate |
Resizes running pods in place where possible, otherwise evicts. |
InPlace |
Only resizes in place, behind a feature gate; defers when it cannot. |
The VPA project warns against using VPA and HPA on the same metric (CPU or memory): VPA raises requests, which lowers utilisation, which makes the HPA scale in, and the two fight.
A concrete example¶
sample-app has a CPU request of 250m, an HPA targeting 60% average utilisation, minReplicas: 2, maxReplicas: 10.
- 4 pods are running, averaging 90% of their request (225m each).
ceil(4 × 90 / 60) = ceil(6.0) = 6. The ratio 1.5 is outside the tolerance, so the HPA sets 6 replicas.- Two new pods are Pending because the nodes are full. Karpenter sees unschedulable pods and launches a node (Karpenter in depth); about a minute later they run.
- Load drops to 30%. The HPA would now want
ceil(6 × 30 / 60) = 3, but the 5-minute scale-down window holds 6 until the lower recommendation has lasted that long. - Total reaction time to a spike is metric lag plus pod startup plus, if needed, node startup. For sudden traffic, keep headroom (a higher
minReplicasor a lower target) rather than relying on autoscaling alone.
Why not just...?¶
- ...set a high fixed replica count? It works, and for small, steady services it is often the right answer. It costs money at night and still runs out at an unusual peak.
- ...scale on CPU for everything? A queue worker can be idle on CPU with a million messages waiting, and a web app can be limited by a database while its CPU is low. Scale on the signal that measures the work: queue length, requests per second, or latency.
- ...use the HPA and keep
replicas:in the Deployment manifest? Every apply or GitOps sync resets the count, fighting the HPA. Removespec.replicasfrom the manifest once an HPA owns it (Sync, hooks and waves).
Commands to know¶
An HPA on a cluster without a metrics server:
$ kubectl autoscale deployment hpa-demo -n demo --min=1 --max=5 --cpu=50%
horizontalpodautoscaler.autoscaling/hpa-demo autoscaled
$ kubectl get hpa hpa-demo -n demo
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
hpa-demo Deployment/hpa-demo cpu: <unknown>/50% 1 5 1 0s
$ kubectl describe hpa hpa-demo -n demo
...
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
$ kubectl top pods -n demo
error: Metrics API not available
<unknown> in TARGETS and ScalingActive False mean the HPA has no numbers and will not scale. --cpu=50% is the current flag; it writes type: Utilization, averageUtilization: 50. The generated spec:
spec:
maxReplicas: 5
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50
minReplicas: 1
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: hpa-demo
| Command | What it does |
|---|---|
kubectl get hpa -A |
Every HPA, its current and target values, and replica counts. |
kubectl get hpa <name> -w |
Watch decisions as load changes. |
kubectl describe hpa <name> |
Conditions (AbleToScale, ScalingActive, ScalingLimited) and events with the reason for each change. |
kubectl top pods -n <ns> |
Current CPU and memory from the metrics API. |
kubectl get apiservice v1.metrics.k8s.io |
Whether the metrics API is registered and available. |
kubectl get --raw /apis/metrics.k8s.io/v1/namespaces/<ns>/pods |
The raw numbers the HPA sees. |
How it breaks¶
| Symptom | Layer | Check | Likely cause |
|---|---|---|---|
TARGETS shows <unknown> |
Metrics | kubectl top, kubectl get apiservice |
No metrics-server, or its APIService is unavailable |
| Events say a container has no CPU request | Pod spec | Container resources.requests |
Utilisation target on a container with no request |
| Replica count jumps back after every deploy | Ownership | Manifest spec.replicas, GitOps diffs |
Manifest and HPA both set replicas |
At maxReplicas, still slow |
Bottleneck elsewhere | ScalingLimited condition, database, dependencies |
Limit too low, or the app is not CPU-bound |
| New pods stay Pending | Nodes | kubectl describe pod events |
Node autoscaler missing, at its limit, or no instance type fits |
| Replicas flap up and down | Behaviour | HPA events over time | Short scale-down window, or a noisy metric |
| VPA and HPA keep changing the same app | Conflict | Both objects' targets | Both acting on CPU or memory |
Interview questions¶
1. How does the HPA decide how many replicas to run?
Answer
desired = ceil(current × currentMetric / targetMetric), every 15 seconds by default.- Skips changes within a 10% tolerance; ignores or discounts not-ready pods and missing metrics.
- Applies
behaviorlimits, then sets the target's scale subresource.
2. An HPA shows <unknown> for its target. What do you check?
Answer
kubectl top pods: if it fails, metrics-server or the metrics APIService is the problem.kubectl describe hpa: the condition message names the failure.- Container CPU requests: utilisation targets need them.
3. Why does a CPU utilisation target depend on requests?
Answer
- Utilisation is usage divided by the pod's request.
- No request means no denominator, so the HPA cannot compute it.
- Requests that are far too high or low make the same target mean very different real loads.
4. What are the differences between HPA, VPA and a node autoscaler?
Answer
- HPA: number of pods; core Kubernetes.
- VPA: requests per pod; an add-on with Off, Initial, Recreate and in-place modes.
- Node autoscaler: number of nodes, reacting to unschedulable pods (Cluster Autoscaler, Karpenter).
5. Why is scale-down slower than scale-up by default?
Answer
- Scale-down uses a 300-second stabilisation window: the highest recommendation of the last 5 minutes.
- It prevents flapping when load is spiky; removing capacity too early causes the next spike to hurt.
- Scale-up has no window, and can double or add 4 pods every 15 seconds.
6. A queue worker should scale on queue length, including down to zero. How?
Answer
- Expose queue length as an external metric (an adapter, or KEDA, which manages the HPA for you).
- An HPA with an external metric may use
minReplicas: 0, beta and on by default since 1.37; KEDA has supported it longer. - Accept the cold start: the first message waits for a pod to start.
7. Traffic spiked, the HPA scaled out, but users still saw errors for three minutes. Why, and what would you change?
Answer
- Reaction time adds up: metric collection, 15-second loop, pod startup and readiness, possibly a new node.
- Keep headroom: a higher
minReplicasor a lower target, faster startup, and scheduled pre-scaling before known peaks. - Make sure readiness probes keep traffic off pods until they can serve (Probes and graceful shutdown).
Try it¶
Step 1. On a kind cluster (no metrics server by default), create a Deployment with a CPU request and an HPA.
kubectl create namespace hpa-try
kubectl create deployment w -n hpa-try --image=busybox:1.36 -- sleep 3600
kubectl set resources deployment w -n hpa-try --requests=cpu=100m
kubectl autoscale deployment w -n hpa-try --min=1 --max=5 --cpu=50%
Step 2. Predict what TARGETS shows and which condition is false, then check.
kubectl get hpa w -n hpa-try
kubectl describe hpa w -n hpa-try | grep -A4 Conditions
Step 3. On paper: 3 pods at 80% of request with a 50% target. How many replicas? (Answer: ceil(3 × 80 / 50) = ceil(4.8) = 5.) Then clean up.
kubectl delete namespace hpa-try
Zoom out¶
Autoscaling is a feedback loop like every controller in Kubernetes (Architecture and the reconcile loop), and it is only as good as its inputs: correct requests, a meaningful metric, and fast startup. The node layer is where the cost is (Cost optimisation), and the metric layer is where observability comes in (Prometheus on Kubernetes).
Sources¶
- Kubernetes documentation: horizontal pod autoscaling
- Kubernetes documentation: vertical pod autoscaling
- Kubernetes documentation: autoscaling workloads
- Kubernetes blog: v1.37, scale workloads to zero with the HPA
- Kubernetes blog: v1.37, Metrics API graduates to stable
- Kubernetes autoscaler project: VPA known limitations
- KEDA documentation: concepts