Introduction to Live Streaming Pipeline

1. Streaming and Distribution Principles

A typical live streaming pipeline starts from the capture side, where audio and video are pushed via a streaming protocol to a server or CDN. The CDN then transcodes and repackages the stream into multiple protocol formats (e.g., FLV, HLS, HTTP-FLV, LLS) for viewers to pull and play.

Modern protocols (like SRT, RIST, E‑RTMP) and standard ones (RTMP, HLS, HTTP‑FLV, RTC) serve different purposes: some prioritize ultra-low latency, while others focus on compatibility and large-scale delivery.

• Capture and Push: The broadcaster typically uses RTMP to push streams, but SRT, RIST, or E‑RTMP can also be used. Upstream network quality affects push latency and stability.

  • SRT (Secure Reliable Transport): Based on UDP + ARQ retransmission; supports AES encryption and firewall traversal; typical latency ≈120 ms. Ideal for cross-network, high-reliability transmission.

  • RIST (Reliable Internet Stream Transport): Based on RTP/UDP + NACK; open standard focused on multi-vendor compatibility and low latency.

  • E‑RTMP (Enhanced RTMP): Extends RTMP with support for modern features such as AV1/HEVC, nanosecond timestamps, and multi-track audio.

• Server / Transcoding / CDN: Handles stream distribution and protocol conversion. Each protocol has a different processing model: HLS requires segmenting and indexing (introducing batch latency); RTMP transmits frame by frame (lower latency).

  • Receive protocol stream → Transcode into multiple bitrates (H.264, H.265, AV1, etc.)

  • Output formats on demand: FLV/HTTP‑FLV (second-level latency), HLS/LL-HLS (cross-platform), MPEG‑DASH/LL‑DASH, WebRTC (real-time interaction)

  • CDN handles segment caching (.ts/.m4s), index distribution (.m3u8/.mpd), and edge protocol relay.

• Playback Client: The player downloads, decodes, and renders the stream. Buffer size and decoding delay determine final latency. Players often buffer a few frames or seconds to smooth network fluctuations, which introduces extra delay.

2. Latency Composition and GOP Buffer Strategy

Total live latency is made up of three parts:

1. Production Side: Capture (10–30 ms), encoding (50–200 ms), preprocessing

2. Network/CDN: Transmission (≳ network RTT), segmentation & distribution (HLS segment duration × playlist length)

3. Playback Side: Player buffer (1–3 s), decoding, rendering, post-processing

The GOP (Group of Pictures) value defines the number of frames between keyframes (IDR), i.e., the maximum frame count in a group.

Clients usually buffer 1–2s of content to mitigate jitter. To balance startup speed and latency, two GOP buffer strategies are common:

• Upper-Bound Priority (Low Latency): No historical I-frame buffering; player waits for the next I-frame to begin playback (minimal delay).

• Lower-Bound Priority (Anti-Jitter): Buffers complete GOPs and starts playback immediately using the cached I-frame ("instant start"), at the cost of increased delay (≈1 GOP length).

Using SRS as an example, the gop_cache config toggles this strategy:

• gop_cache off: enables upper-bound priority (low-latency playback),
• gop_cache on: allows fast startup using buffered GOPs.

Toggle based on scenario—enable buffering to avoid black screens in poor networks, or disable to reduce delay when downstream is stable.

3. Protocol Mechanisms and Use Cases

• RTMP (Real-Time Messaging Protocol): Developed by Adobe/Flash, uses long-lived TCP connections to transmit FLV packets. Pros: Low latency (1–3s), widely supported by capture devices and legacy players, firewall-friendly (often via port 80). Cons: Requires plugins or dedicated players (modern browsers don’t support Flash), TCP may cause accumulated delay under poor network. Use Cases: Interactive streams, PC-based streaming.

• HTTP-FLV: Streams FLV via continuous HTTP requests. Works similarly to RTMP—packages audio/video into FLV Tags and transmits over HTTP.

  • Pros: Uses standard HTTP (bypasses firewall), supports HTTPS, mobile-friendly.

  • Cons: Still based on TCP, requires a player or JS engine with HTTP-FLV support (not playable natively in browsers).

  • Use Cases: Web-based live streaming.

• HLS (HTTP Live Streaming): Apple’s segmented streaming protocol over HTTP. Server slices video into TS or CMAF segments and generates an M3U8 playlist.

  • Pros: Fully HTTP-based, firewall-traversable, CDN-friendly, simple client implementation.

  • Cons: Intrinsically high latency due to segment-based delivery; standard delay typically 5–15s. LL-HLS reduces latency to 3–5s via shorter segments and partial loading.

  • Use Cases: Video-on-demand and general-purpose live streaming.

• LLS (Low-Latency Streaming): Refers to proprietary low-latency solutions using fine-grained segments (e.g., CMAF/FLV hybrid) and push-based delivery.

  • Pros: Sub-second latency, real-time delivery at scale.

  • Cons: Vendor-locked, requires specific SDKs/players, higher deployment cost.

  • Use Cases: Large-scale live events requiring low latency.

• RTC (WebRTC): UDP-based real-time communication protocol using SRTP.

  • Pros: Ultra-low point-to-point latency (<500ms), ideal for video conferencing or co-hosting.

  • Cons: Requires complex signaling and NAT traversal; not natively scalable (requires SFU/MCU); browser and hardware compatibility varies.

  • Use Cases: 1-on-1 or small group low-latency interactions; not suitable for CDN broadcasting.

4. Transcoding Profiles

Encoder Options:

• H.264: Most widely compatible, suitable for SD/HD.

• H.265: Higher compression (≈50% bitrate reduction at same quality), suitable for 4K/8K, but requires client support.

For users on poor networks, a Narrowband HD profile may be offered—maintains resolution while reducing bitrate.

Example transcoding profiles (actual strategy may vary by vendor):