Traffic management¶
Status: AI draft, 2026-11-16. Not yet verified by me. Written from the Istio documentation; Istio was not installed for these notes.
In one line¶
In Istio a Gateway admits traffic, a VirtualService decides where each request goes (path and header matches, weights, retries, timeouts, faults), and a DestinationRule defines subsets and how to reach them (load balancing, connection limits, outlier detection).
The plain idea¶
A Kubernetes Service spreads requests evenly over all its pods (Services and DNS). That is not enough to say "send 10% of traffic to version 2", "send requests with header x-beta: true to the new build", "give up after 2 seconds and retry once", or "stop sending traffic to a pod that keeps failing". Istio's traffic resources add those decisions to every proxy, so they apply to every caller without code changes.
How it works¶
The resources¶
flowchart TD
C["Client outside the mesh"] --> GW["Gateway: ingress proxy listening<br/>on 443 for shop.example.com"]
GW --> VS["VirtualService: match /api, split<br/>90 to v1 and 10 to v2, retries, timeout"]
M["Caller inside the mesh"] --> VS
VS --> DR["DestinationRule: subsets v1 and v2 by pod label,<br/>load balancing, connection pool, outlier detection"]
DR --> P1["Pods labelled version: v1"]
DR --> P2["Pods labelled version: v2"]
| Resource | Answers | Key fields |
|---|---|---|
| Gateway (Istio API) | Which ports, hosts and certificates an ingress or egress proxy serves | servers with port, hosts, TLS |
| VirtualService | Where does a request to this host go? | hosts, gateways, http routes with match, route (destination, subset, weight), retries, timeout, fault, mirror |
| DestinationRule | How is traffic sent to this destination once chosen? | subsets (by labels), trafficPolicy: loadBalancer, connectionPool, outlierDetection, TLS mode |
| ServiceEntry | Adds external services to the mesh registry | Hosts, ports, resolution |
Retries, timeouts and faults¶
- Timeouts on HTTP routes are disabled by default; set them per route.
- Retries: if a route sets none, the mesh default applies: 2 attempts on
connect-failure,refused-stream,unavailableandcancelled(customisable in mesh config). SetperTryTimeoutso retries fit inside the overall timeout, and only retry idempotent operations (High availability and failure domains). - Fault injection adds delays or aborts for a share of requests, to test how callers cope; while faults are enabled on the client side, that route's timeouts and retries are not applied.
Outlier detection and connection pools¶
- Outlier detection (passive health checking): an endpoint that returns consecutive 5xx errors is ejected from the load-balancing pool for a period, which works like a circuit breaker per pod.
- Connection pool limits cap connections and pending or concurrent requests to a destination; excess requests fail fast instead of piling up.
A canary by weight¶
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
subsets:
- name: v1
labels: { version: v1 }
- name: v2
labels: { version: v2 }
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
name: reviews
spec:
hosts: [reviews]
http:
- match:
- headers:
x-beta: { exact: "true" }
route:
- destination: { host: reviews, subset: v2 }
- route:
- destination: { host: reviews, subset: v1 }
weight: 90
- destination: { host: reviews, subset: v2 }
weight: 10
timeout: 3s
retries:
attempts: 2
perTryTimeout: 1s
Testers send x-beta: true to reach v2 every time; everyone else gets a 90/10 split. Moving to 50/50 or 100/0 is an edit to two numbers, often driven by a progressive delivery tool (Deployment strategies).
How it maps to Envoy¶
| Istio | Envoy concept |
|---|---|
| Gateway servers, service ports | Listeners (where traffic arrives) |
| VirtualService routes | Routes (match, then choose a cluster, with retries and timeouts) |
| Destination host plus subset, DestinationRule policies | Clusters (a group of endpoints and how to connect: load balancing, TLS, pools, outlier detection) |
| Pod IPs from Kubernetes | Endpoints |
istioctl proxy-config listeners|routes|clusters|endpoints <pod> shows exactly this, which is the main debugging tool (Mesh troubleshooting).
Kubernetes Gateway API¶
Istio implements the Kubernetes Gateway API (Gateway, HTTPRoute) for ingress and, with GAMMA, for traffic inside the mesh, and states it intends to make Gateway API the default traffic management API in the future (Ingress and Gateway API). With Gateway API, Istio can also deploy the gateway proxies automatically from a Gateway resource.
A concrete example¶
sample-app's checkout service rolls out v2:
- v2 pods deployed with label
version: v2; DestinationRule subsets for v1 and v2, with outlier detection (eject after 5 consecutive 5xx for 30 seconds). - VirtualService: internal testers by header to v2; everyone else 95/5.
- Error rate and p99 latency per subset come from the proxies' metrics; at 95/5 for an hour without regressions, the weight moves to 75/25, then 50/50, then 0/100 (SLOs and error budgets).
- Timeouts of 3 seconds with 2 retries for the idempotent
GETroutes only; no retries onPOST /orders.
Why not just...?¶
- ...do canaries with replica counts? 1 of 10 pods gives roughly 10%, but only in steps of 10%, and header-based routing is impossible. Weighted routes decouple traffic share from pod count.
- ...retry everything three times? Retries multiply load during an outage and can repeat non-idempotent operations. Retry only safe requests, with budgets and per-try timeouts.
- ...configure retries both in the app and the mesh? They multiply (3 × 3 = 9 attempts). Choose one layer.
Commands to know¶
These were not run for this page.
| Command | What it does |
|---|---|
kubectl get virtualservices,destinationrules,gateways -n team-a |
Traffic resources in a namespace. |
istioctl analyze -n team-a |
Detects common mistakes (subsets with no matching pods, missing hosts). |
istioctl proxy-config routes deploy/frontend -n team-a |
Routes as the caller's proxy sees them. |
istioctl proxy-config clusters deploy/frontend -n team-a --fqdn reviews.team-a.svc.cluster.local |
Clusters (subsets) for a destination. |
istioctl proxy-config endpoints deploy/frontend -n team-a |
Endpoints and their health for every cluster the proxy knows. |
istioctl x describe pod <pod> |
Which VirtualServices and DestinationRules apply to a pod. |
To see one subset only, filter by cluster name, which has the form outbound|9080|v2|reviews.team-a.svc.cluster.local: add --cluster "outbound|9080|v2|reviews.team-a.svc.cluster.local" (quoted, because of the pipes).
How it breaks¶
| Symptom | Layer | Check | Likely cause |
|---|---|---|---|
503 with NR (no route) |
Routing | VirtualService hosts and matches | No route matches, or host name mismatch |
503 UH for a subset |
Subsets | Pod labels vs subset labels | Subset selects no pods (Mesh troubleshooting) |
| Weights ignored | Scope | Which proxy applies the rule | VirtualService not bound to the gateway or mesh, or traffic bypasses the mesh |
| Requests time out at odd durations | Timeouts | Route timeout vs app timeouts | Mesh and app timeouts or retries stacking |
| Healthy pods getting no traffic | Outlier detection | Ejection stats | Too aggressive ejection thresholds |
| Duplicate side effects | Retries | Retry policy on non-idempotent routes | Retries on POST |
Interview questions¶
1. What do VirtualService and DestinationRule each do?
Answer
- VirtualService: routing decisions: matches, weights, retries, timeouts, faults.
- DestinationRule: what the destinations are (subsets) and how to connect (load balancing, pools, outlier detection, TLS).
- Routing chooses a subset; the DestinationRule defines it.
2. How do you run a canary with Istio?
Answer
- Label pods by version; define subsets in a DestinationRule.
- VirtualService with weighted routes (and optionally header matches for testers).
- Shift weights while watching per-subset metrics.
3. What is outlier detection?
Answer
- Passive health checking: endpoints with consecutive errors are ejected from load balancing for a period.
- Protects callers from a bad pod without waiting for readiness probes.
- Tune thresholds so healthy pods are not ejected by brief errors.
4. What are Istio's default retries and timeouts?
Answer
- Route timeouts are disabled by default.
- The default retry policy is 2 attempts on connect failures, refused streams, unavailable and cancelled.
- Set explicit per-route values, and avoid retries on non-idempotent requests.
5. How do Istio resources map to Envoy configuration?
Answer
- Gateways and ports become listeners; VirtualServices become routes.
- Destinations and subsets with DestinationRule policies become clusters; pods become endpoints.
istioctl proxy-configshows each.
6. A canary at 10% receives no traffic. What do you check?
Answer
- Do v2 pods carry the subset's labels, and are they ready?
- Is the VirtualService applied to the caller's traffic (host names, gateway binding)?
istioctl proxy-config routesandendpointson a caller.
Try it¶
Without installing Istio:
- Write a VirtualService that sends
/api/v2/*to subset v2 and everything else to v1, with a 2-second timeout. - Write a DestinationRule that ejects a pod after 5 consecutive 5xx errors for 30 seconds and caps concurrent requests at 100.
- Rewrite the canary above as a Gateway API
HTTPRoutewith weightedbackendRefs(Ingress and Gateway API).
Zoom out¶
Traffic management is the mesh's version of deployment strategies and resilience patterns, applied at every hop (Deployment strategies, High availability and failure domains). Security policies use the same proxies (mTLS and authorization).