Patterns for low-latency media paths

What Is a Media Path?

In real-time communication, the media path is the route audio and video data takes from the sender's device to the receiver. This is distinct from the signaling path, which handles call setup and control messages.

The media path determines the latency, quality, and reliability of the actual communication experience. A well-designed media path minimizes the number of hops, keeps processing close to endpoints, and adapts to changing network conditions in real time.

The Components of End-to-End Media Latency

Pattern 1: Direct Peer-to-Peer Paths

For two-party communications, the lowest possible latency is achieved when media flows directly between endpoints without traversing any intermediate server. WebRTC ICE (Interactive Connectivity Establishment) attempts peer-to-peer connection first, falling back to TURN relay only when direct connection is impossible.

Direct paths eliminate the server processing latency component entirely and reduce geographic routing overhead. In favorable network conditions, direct peer-to-peer WebRTC achieves 30-60ms end-to-end latency.

Optimize your media path for sub-500ms latency

Request a Audit

Pattern 2: Geographically Distributed TURN and SFU

When direct peer-to-peer paths are not possible, relay through TURN servers or media through SFU nodes is required. The critical design decision is server placement. A TURN server in Singapore adds minimal latency for users in Southeast Asia. The same TURN server adds 200+ms for users in Western Europe.

Low-latency architectures require TURN and SFU nodes deployed in every region where users are located. The client should connect to the nearest available node, not a centralized server.

Pattern 3: Codec Selection for Latency

Different audio codecs introduce different amounts of algorithmic delay. The codec's frame size determines the minimum packetization delay. Opus codec at 10ms frame size is optimal for low-latency voice applications. G.729 with 30ms frames adds unnecessary delay for latency-sensitive scenarios.

• Opus at 10ms frame: ~20ms algorithmic delay, excellent for voice AI

• G.711 at 20ms frame: ~20ms algorithmic delay, good PSTN compatibility

• G.729 at 30ms frame: ~35ms algorithmic delay, higher compression

• Avoid comfort noise generation (CNG) algorithms that add jitter-equivalent behavior

Pattern 4: Adaptive Jitter Buffering

Jitter buffers add controlled delay to smooth out network-induced packet arrival variation. The tradeoff is that larger jitter buffers reduce audio glitches but add latency. Adaptive jitter buffers measure actual network jitter in real time and adjust their size accordingly.

A well-tuned adaptive jitter buffer stays at 20-30ms during stable network conditions and expands to 60-80ms only when measured jitter requires it. Systems using fixed large jitter buffers sacrifice latency unnecessarily.

Pattern 5: SIP Media Path Optimization

SIP-based systems have additional optimization opportunities in the SDP negotiation phase. Direct media (no media proxy) configurations allow endpoints to establish direct RTP paths without routing through the SIP server. This requires NAT handling at the endpoint level but eliminates server-side media processing latency.