The idea: a proxy next to every pod

Once a system is split into many services, every call between them needs the same things: encryption, a check of who is calling, load balancing, retries, timeouts, and a way to stop calling a pod that is broken. A service mesh takes all of that out of the apps. Each pod gets a second container, a proxy called the sidecar (in Istio it is Envoy). The apps keep speaking plain HTTP to names like reviews:9080; the sidecars do the rest. The proxies together are the data plane. A control plane, istiod, tells every proxy what to do and gives each one a certificate.

istiod above two pods; each pod has an app and an Envoy sidecar; the apps talk plain HTTP to their own sidecar, the sidecars talk mutual TLS to each other, and istiod sends config and certificates down to both sidecars over xDS.
The apps only talk to their own sidecar; the sidecars talk mutual TLS to each other, and istiod configures them all.

The page has four pods in the namespace shop: frontend, two replicas of reviews, and a legacy pod that has no sidecar. Every request goes from an app to reviews:9080 (Demo 1: one request through two sidecars).

How traffic gets into the sidecar: iptables

The app doesn't know the proxy is there, so something has to steer its packets. When a pod is created, Istio injects an init container (istio-init, or the Istio CNI plugin) that writes iptables rules into the pod's network namespace. The pod's containers share that namespace, so the rules apply to the app too:

  • outgoing TCP (nat OUTPUT → ISTIO_OUTPUT) is REDIRECTed to 127.0.0.1:15001, Envoy's outbound listener;
  • incoming TCP (nat PREROUTING → ISTIO_INBOUND) is redirected to port 15006, Envoy's inbound listener;
  • traffic from Envoy itself (it runs as uid 1337) is let through, or it would loop back into itself.

So one call crosses two proxies: the caller's sidecar (outbound) and the callee's sidecar (inbound). Envoy reads the original destination from the redirected socket (SO_ORIGINAL_DST) and the Host header, picks a route and a cluster, and load-balances over the cluster's endpoints itself, so kube-proxy's Service IP is not used.

Without a mesh the app's connection goes straight to the other pod. With a mesh, iptables redirects it to Envoy port 15001, Envoy sends mutual TLS to the other pod, where iptables redirects it to Envoy port 15006, which forwards to the app on localhost.
iptables bends the app's connection through two Envoys: 15001 on the way out, 15006 on the way in.

Mutual TLS and identities

Between the sidecars the connection is mutual TLS: both sides show a certificate. The certificate does not name a host but a workload identity in SPIFFE form, built from the pod's service account: spiffe://cluster.local/ns/shop/sa/frontend. istiod is the certificate authority: the agent in each sidecar asks for a certificate, and Envoy gets it over SDS. Certificates live about a day and are rotated without anyone noticing. The receiving sidecar can then apply an AuthorizationPolicy such as "only sa/frontend may call reviews".

A PeerAuthentication resource sets the mode. PERMISSIVE (Istio's default) accepts both mutual TLS and plain text, which helps while pods are being moved into the mesh. STRICT accepts only mutual TLS: the legacy pod's plain request fails the handshake at the inbound sidecar and never reaches the app (Demo 2: a pod without a sidecar is refused). See Mutual TLS for the handshake itself.

Retries and timeouts in the proxy

A VirtualService sets the retry policy and the timeout of a route. They run in the caller's sidecar, so a retried failure never reaches the app. The page's policy is retries: {attempts: 2, perTryTimeout: 1s, retryOn: 5xx} and timeout: 2.5s.

casetriesthe app sees
reviews-2 answers 503, no retriesreviews-2 503503 after 5 ms
reviews-2 answers 503, retriesreviews-2 503 → reviews-1 200200 after 55 ms
reviews-2 slow (5 s), retriesreviews-2 timeout at 1 s → reviews-1 200200 after 1050 ms
all pods slow, retriestimeout, timeout, cut at 2.5 s504 after 2500 ms
all pods slow, no timeoutone try200 after 5000 ms

The per-try timeout lets a retry go to a healthy pod instead of waiting on a stuck one; the route timeout caps the whole call, retries included (Demo 4: timeouts). The page's starting config (v1) turns retries off with attempts: 0; without any VirtualService, Istio already retries some failures (such as a refused connection) twice. Retries multiply load: with 2 retries a sick service can get 3 times its normal traffic, so retry only what is safe to repeat (idempotent requests) and keep budgets small. Istio sets no route timeout unless you ask for one.

Circuit breaking: outlier detection

A DestinationRule configures the cluster on the caller's side. Its outlier detection watches each endpoint's answers: consecutive5xxErrors: 2 ejects a pod that failed twice in a row, for baseEjectionTime: 30s times the number of times it was ejected. While ejected, the load balancer skips it, so requests stop paying for a failed try first (Demo 5: outlier detection ejects a pod). maxEjectionPercent keeps part of the pool in service. The other half of Istio's circuit breaking is the connectionPool limits (http1MaxPendingRequests, maxRequestsPerConnection, …): when the queue to a cluster is full, Envoy answers 503 at once instead of piling up more work. This is the same idea as the circuit breaker and bulkhead in a library like Resilience4j, done in the proxy for every language at once.

The control plane and xDS

Nobody edits Envoy's config by hand. istiod watches the Kubernetes API (Services, endpoints, VirtualServices, DestinationRules, …), turns them into Envoy config and streams it to every sidecar over xDS, a family of gRPC discovery APIs. Istio sends them all on one stream (ADS) per sidecar:

APIcarrieson the page
LDS (listeners)ports to accept on: 15001, 15006LDS row
RDS (routes)host + path → cluster, retries, timeoutsfrom the VirtualService
CDS (clusters)load-balancing policy, outlier detection, connection pools, TLSfrom the DestinationRule
EDS (endpoints)pod IPs behind each clusterreviews-1, reviews-2
SDS (secrets)the workload certificate and the root CAsa/frontend

A change is pushed, not polled: istiod sends a new version, each Envoy loads it while running and answers with an ACK (a request carrying the version and the response's nonce). A config it can't load gets a NACK and the old one stays in use (Demo 3: istiod pushes retries over xDS). If istiod goes down, the sidecars keep running with the last config they got; only changes stop.

Mesh, library, gateway

in-app libraryservice meshAPI gateway
whereinside each appa proxy next to each podone proxy at the edge
trafficthe calls the app makesservice to service (east-west)client to service (north-south)
languagesone library per languageanyany
costcode in every servicetwo extra hops, memory per podone extra hop

An API gateway handles requests coming into the system; the mesh handles calls inside it, and the two are often used together. Underneath, the packets still travel the pod network shown in the life of a request in Kubernetes. Istio's newer ambient mode drops the per-pod sidecar: a per-node proxy (ztunnel) does mutual TLS, and an optional waypoint proxy does the HTTP features.

What the page leaves out

Real Envoy waits a jittered back-off (25 ms base) between retries and picks among all hosts, not a strict round robin; outlier detection also runs on a timer (interval) and has success-rate and gateway-error variants. The page skips the sidecar's startup (certificate signing request, the first xDS fetch), locality and weighted routing (canary v1/v2 splits), AuthorizationPolicy, telemetry (every sidecar reports metrics and trace spans), HTTP/2 between sidecars, and the Sidecar resource that limits which config each proxy gets.