The idea: trade a little delay for smooth audio
In a voice call the sender's codec cuts the sound into frames of 20 ms and sends each one as an RTP packet over UDP, one packet every 20 ms. The speaker at the other end must also play one frame every 20 ms, with no gaps. The network does not keep the spacing: each packet waits in different router queues, so one takes 40 ms and the next 90 ms. That variation is called jitter. Packets can also arrive out of order or not at all.
The receiver's jitter buffer holds packets for a while before playing them. It puts them back in order by their sequence number and plays one frame every 20 ms at a fixed offset from when it was sent. The page tracks 16 packets (#0 to #15) through the sender, the network and the buffer, then shows what the speaker played and a chart of each packet's delay.
RTP: sequence number and timestamp
Each RTP header (RFC 3550) carries a sequence number, one higher per packet, and a timestamp in samples of the media clock. At 8 kHz, 20 ms is 160 samples, so packet #n has timestamp 160 n (real streams start both at random values). The sequence number lets the receiver put packets back in order and spot gaps; the timestamp tells it when, relative to the first frame, each frame should play. RTP runs over UDP because a retransmitted frame would come too late to be useful (see TCP vs UDP).
Playout delay and late packets
When the first packet arrives, the receiver waits B ms (the buffer delay) and plays it, then plays one frame every 20 ms. So frame #n plays at the same distance from its send time for every n: the playout delay, here the first packet's network delay plus B. On the chart that is the black line over each column. A packet whose network delay (the bar) is taller than the line arrives after its frame's play time: it is late, and the receiver throws it away. For the speaker a late packet is the same as a lost one.
| Packet arrives… | What the receiver does |
|---|---|
| before its play time, in order | stores it in its slot, plays it on time ✓ |
| before its play time, after a later packet | out of order: the buffer still slots it by sequence number ✓ |
| after its play time | late: discarded ✗; its frame was already concealed |
| never | lost: concealed ✗ |
Demo 1: steady network has no jitter, so a 20 ms buffer is enough. Demo 2 runs the jittery network with the same 20 ms buffer: packets #2, #5, #6, #8 and #12 come later than the 60 ms playout delay and become glitches.
The trade-off: latency vs glitches
A deeper buffer catches slower packets, but every frame then plays later. Demo 3 runs the same jittery network with a 100 ms buffer: all 16 frames play on time, but the playout delay grows from 60 to 140 ms. For a conversation, ITU-T G.114 suggests keeping the one-way mouth-to-ear delay under about 150 ms; above that people start talking over each other. The buffer is one part of that budget, next to the codec, the network and the sound card.
Packet loss concealment
When a frame is missing at its play time the decoder cannot stay silent: a 20 ms hole is a loud click. Packet loss concealment (PLC) plays a guess instead: the simplest repeats the last frame with a fade; codecs such as Opus and G.711 Appendix I extend the pitch of the last good audio. One missing frame is hard to hear; several in a row sound robotic and then fade to silence. Demo 4: packet loss loses #5, #8 and #9; the buffer is big enough, so every other frame plays, and the gaps are concealed. No buffer size can bring back a lost packet.
The adaptive buffer
A fixed buffer is either too small when the network gets worse or too big when it is calm. An adaptive jitter buffer (WebRTC's NetEq, for example) changes B during the call. The page uses a simple rule:
- Grow: a late packet makes the buffer grow by enough that this packet would have fit, rounded up to 20 ms. The speaker plays 20 ms of stretched audio per step (time stretching, as NetEq's "expand"), and every later frame plays 20 ms later.
- Shrink: after 3 packets in a row arrive with 40 ms or more to spare, the buffer drops 20 ms by squeezing two frames into one 20 ms step (NetEq's "accelerate").
Demo 5 sends a delay spike: #3 to #6 take 90 to 140 ms. The buffer starts at 20 ms, so those four frames are concealed; the late packets make it grow to 100 ms (four stretch steps), and once the network is calm again two squeezes bring it back down to 60 ms. Real buffers change size mostly during silence, where a stretched or shortened pause is not heard.
For how queueing delay builds up in the first place, see Latency vs Throughput.
What the page leaves out
Real receivers do not know the send time: they estimate jitter from RTP timestamps and arrival times (the interarrival jitter of RFC 3550, a running average) and pick the target delay from a delay histogram, not from single late packets. The page uses fixed delay lists, one stream, 20 ms ticks and a first packet that is never late. It leaves out RTCP reports, forward error correction and redundancy (Opus in-band FEC, RED), retransmission (NACK) for video, clock drift between the two sound cards, silence suppression and comfort noise, lip sync between audio and video, and video jitter buffers, which hold whole frames made of many packets.