Site logo WebRTC & Real-Time Media Architecture

SFU vs MCU Topologies

Once a call grows past a handful of participants, the peer-to-peer mesh that powers two-party WebRTC collapses under its own uplink: every participant must encode and send a separate stream to every other participant, so an N-party mesh forces each browser to run N-1 encoders and N-1 uploads. The participant count at which that stops being viable is worked through in Mesh vs SFU: When to Graduate. A central media server fixes this, and the two classic server topologies are the Selective Forwarding Unit (SFU) and the Multipoint Control Unit (MCU). This guide is part of the Media Server Architecture: SFU & MCU guide, and its goal is to give you a precise, decision-ready model of how each topology ingests, processes, and emits media so you can pick the right one before you write a line of server code.

The distinction is not academic. It determines your per-participant server CPU cost, your downlink bandwidth bill, the decode load you place on mobile clients, and how much layout flexibility you can offer. The four steps below trace the same media packet through each topology — ingest, processing, emit, and verification — and the comparison table and edge-case notes that follow turn those mechanics into operational guidance.

MCU single mixed stream versus SFU N forwarded streams On the left an MCU decodes three inbound streams, composites and re-encodes them into a single mixed stream sent to each participant. On the right an SFU forwards the three original encoded streams unchanged, so each participant receives N minus one separate streams. MCU — mix and re-encode SFU — forward unchanged Peer A Peer B Peer C MCU decode + composite + re-encode 3 in 1 mixed each Peer A Peer B Peer C SFU route only N-1 each
An MCU decodes all inputs and emits one composited stream per participant; an SFU forwards the original encoded streams untouched, so each participant receives N-1 separate decodes.

Step 1 — How each topology ingests media

Ingest looks identical on the wire and diverges immediately afterward. In both topologies every participant establishes one RTCPeerConnection to the server, completes ICE and the DTLS-SRTP handshake, and pushes encoded RTP exactly as it would to a peer. The server is, from the browser’s perspective, just another ICE agent — which is why the same ICE Candidate Gathering & Filtering rules and TURN fallback apply, and why you still budget 8–20% of sessions onto relays.

The divergence is what the server does with the inbound SRTP. An SFU terminates SRTP, decrypts to plaintext RTP, inspects headers, and stops there — it never touches the encoded payload. It reads the RTP header, the transport-wide-cc feedback, and (for layered media) the simulcast/SVC markers, but the VP8 or H.264 bitstream itself is opaque cargo. An MCU terminates SRTP and then fully decodes every inbound stream to raw YUV frames and PCM samples. That decode is the expensive, defining act of an MCU: a single 720p30 VP8 decode costs real CPU, and the MCU pays it once per inbound stream per room.

// SFU ingest: terminate SRTP, read headers, keep payload encoded
function onInboundRtp(packet, participant) {
  const header = parseRtpHeader(packet);          // ssrc, seq, ts, marker
  const layer  = readSimulcastLayer(packet);      // rid: 'q' | 'h' | 'f' — no decode
  participant.tracks.set(header.ssrc, { header, layer, payload: packet.payload });
  // payload stays VP8/H.264-encoded; the SFU never owns a decoder
  routeToSubscribers(participant, header.ssrc, layer);
}

What the SFU does read closely is the RTP header extension block, and that is where most of its per-packet work actually lives. Three extensions matter: the MID that identifies which bundled m-line a packet belongs to, the RID that names the simulcast layer, and the transport-wide sequence number that feeds congestion control. All three are negotiated per RTCPeerConnection, and the numeric extension IDs are chosen independently by each browser — Chrome may map transport-wide-cc to ID 3 on the publisher’s session and ID 5 on the subscriber’s. A forwarding server therefore cannot copy the extension block verbatim; it has to renumber, and sometimes re-pack, every extension header before the packet leaves, which is the fiddly work broken down in Rewriting RTP Header Extensions When Forwarding. Get it wrong and the symptom is not a crash but a subscriber whose bandwidth estimate never converges, because its feedback references sequence numbers the sender never issued.

The second, quieter divergence at ingest is buffering. An SFU deliberately runs no jitter buffer: it forwards packets close to arrival order and leaves reordering, loss concealment and playout timing to the receiving browser’s own video jitter buffer and NetEq. An MCU cannot do that. Compositing requires frames from different publishers to be aligned on one presentation clock, and RTP timestamps from separate senders are unrelated — each publisher picks a random starting offset, and audio and video use different clock rates (48 kHz and 90 kHz). The MCU must maintain an RTCP sender-report mapping from each publisher’s RTP clock to wall time, then hold 50–150 ms of frames so it can pick the nearest-neighbour frame per input when the compositor ticks. That buffer is not overhead the implementation can optimise away; it is a structural requirement of mixing, and it is the first component of the MCU’s added delay.

Step 2 — How each topology processes media

This is where cost is decided. The SFU’s “processing” is a routing decision, not a media transform: for each subscriber it picks which of a sender’s encoded streams or layers to forward, rewrites RTP sequence numbers and timestamps so the subscriber sees one continuous stream, and translates RTCP feedback (PLI, NACK, REMB/transport-wide-cc) between sender and receiver — the policy that decides when a forwarded PLI actually reaches the publisher is covered in Keyframe Request Strategies in an SFU. It does zero pixel work. When senders publish multiple resolutions, the SFU’s job becomes choosing the right layer per subscriber — the mechanics of which are covered in Simulcast-Aware Forwarding and the per-subscriber bandwidth logic in Bandwidth-Aware Layer Selection in an SFU.

The MCU runs a full media pipeline per room: decode every inbound video to raw frames, composite them onto a canvas according to a layout (grid, active-speaker, presentation), mix every inbound audio track into a single PCM bus with gain control and echo suppression, then re-encode the composite to one output stream. Because the output is a freshly encoded single stream, the MCU can transcode codecs and resolutions freely — it can accept VP8 from one client and emit H.264 to another — which is also why it can serve a SIP endpoint or a dial-in phone bridge that an SFU cannot. The price is that this pipeline is unavoidable and scales with participant count: more inputs means more decodes and a heavier composite per output frame.

// MCU processing: decode all, composite, mix, re-encode once per output layout
function renderMixedFrame(room) {
  const frames = room.participants.map(p => p.decoder.pull());   // N raw YUV decodes
  const canvas = layoutEngine.composite(frames, room.layout);    // grid / active-speaker
  const audio  = audioMixer.sum(room.participants.map(p => p.pcm)); // single PCM bus
  const encoded = room.encoder.encode(canvas);                   // 1 re-encode per layout
  return { video: encoded, audio: audioEncoder.encode(audio) };  // identical for all viewers
}

Laid out against a clock, the two pipelines are not variations of one design: the SFU spends single-digit milliseconds rewriting headers, while the MCU pays four sequential stages before a single frame leaves the server.

Per-frame server pipeline latency: SFU forwarding versus MCU mixing A time axis from zero to 160 milliseconds. The SFU track holds one narrow block, header rewrite and forward, lasting one to three milliseconds. The MCU track holds four consecutive blocks: decode of all inputs about forty milliseconds, composite about thirty-five, audio mix about twenty, re-encode about fifty-five, totalling roughly thirty to one hundred fifty milliseconds of added delay. Per-frame server work, same 720p30 input SFU — forward path rewrite seq / ts, translate RTCP, forward 1–3 ms total, no pixels touched MCU — mix path decode N inputs ≈40 ms composite ≈35 ms audio mix ≈20 ms re-encode composite ≈55 ms added end-to-end delay 30–150 ms, paid on every output frame 0 40 ms 80 ms 120 ms 160 ms Only the MCU stages scale with participant count; the SFU block is flat in N.
The SFU's per-frame work is a header rewrite measured in single-digit milliseconds; the MCU's four-stage decode–composite–mix–re-encode pipeline is what adds 30–150 ms.

Why MCU cost climbs faster than the participant count

It is tempting to model the MCU as N decodes plus one encode and conclude that the encode amortises beautifully across the room. It does not, for two reasons that only show up under load.

The first is that the composite step is not free and does not stay constant. Compositing means scaling every decoded frame to its tile size and blitting it into an output canvas, so the pixel throughput of the compositor is roughly the output resolution times the frame rate — but the scaler cost is proportional to the total input pixels. A 4-way grid at 720p inputs pushes about 3.7 million source pixels per frame through the scaler; a 16-way grid at the same input resolution pushes 14.7 million, even though the output canvas never changed size. Software mixing at 720p30 typically costs on the order of 0.3–0.5 of a CPU core per decoded input and roughly one full core for the x264-class output encode, so a 16-core node comfortably mixes one busy 8–10 person room and no more. The same node forwarding instead of mixing handles several hundred streams, because forwarding is memory bandwidth and syscalls rather than arithmetic on pixels.

The second reason is rate control. An SFU serves every subscriber a stream whose bitrate was chosen by the publisher’s encoder, and adapts per subscriber by switching which layer it forwards — the publisher’s congestion controller and the SFU’s layer picker work independently. An MCU has exactly one encoder per layout, so it has exactly one bitrate for every subscriber sharing that layout. When one participant on a congested mobile link signals a 400 kbps ceiling, the MCU has three choices: encode the shared output at 400 kbps and degrade everyone, drop that participant to audio-only, or spawn a second encode for them. The third option is the one teams reach for, and it is how an MCU deployment silently drifts from one encode per room to one encode per participant — at which point it costs more CPU than an SFU and still adds the mixing latency. Deciding that ceiling explicitly, rather than discovering it at 200 concurrent rooms, is the difference between a capacity plan and a firefight.

Step 3 — How each topology emits media

The SFU emits N-1 distinct encoded streams to each participant (or fewer, if a client subscribes to a subset). Server-side egress is therefore O(N²) in the worst case: a 10-party call with every participant subscribed to every other is up to 90 forwarded streams. The server bandwidth cost is real, but the CPU cost stays near zero because nothing is re-encoded. Each receiving client runs up to N-1 decoders, which is exactly where mobile decode limits bite — most phones cap concurrent hardware video decoders at 1–3.

The MCU emits one stream per participant — or one per distinct layout. Every viewer who wants the same grid receives the same encoded output, so a passive audience of thousands can share a single encode. The receiving client runs exactly one decoder regardless of room size, which is the MCU’s headline advantage for low-power and embedded endpoints. The cost moved server-side: every distinct layout is a separate encode, so per-participant custom views (each user seeing a different active-speaker arrangement) erase the single-encode savings and push MCU CPU toward SFU-like egress without the SFU’s zero-encode benefit.

Counting the streams at three room sizes makes the two cost curves explicit: SFU egress grows quadratically while its client decode count grows linearly, and both MCU numbers stay flat.

Egress and decode fan-out by room size for SFU versus MCU A matrix with columns for rooms of three, ten and fifty participants. The SFU row shows server egress streams of six, ninety and two thousand four hundred fifty, and decoders per client of two, nine and forty-nine. The MCU row shows server egress streams of three, ten and fifty, and exactly one decoder per client at every room size. Streams on the wire, fully subscribed room topology N = 3 N = 10 N = 50 SFU — forward server egress streams decoders per client 6 90 2450 2 9 49 MCU — one mix server egress streams decoders per client 3 10 50 1 1 1 SFU egress = N(N-1) and decoders = N-1; MCU egress = N and decoders = 1, whatever N is.
SFU egress and client decode counts scale with room size; MCU egress is linear and every client decodes exactly one stream.

Raw stream counts overstate the SFU’s downlink problem, though, because the streams are not the same size. When publishers send simulcast, the SFU chooses per subscriber and per tile, and the layers sit at 1/2 and 1/4 scale. A 720p top layer runs around 1.5 Mbps; the half-scale 360p layer lands near 600 kbps and the quarter-scale 180p layer near 150 kbps. A 9-person grid where one speaker is shown large and eight are thumbnails therefore costs about 1.5 Mbps + 8 × 150 kbps ≈ 2.7 Mbps down, not the 13.5 Mbps that “nine 720p streams” implies. That is the number to compare against an MCU’s single 1.5–2 Mbps composite, and it is why the SFU’s bandwidth disadvantage is real but far smaller than the naive N-1 count suggests. Whether you buy that adaptivity with simulcast’s parallel encodes or SVC’s single layered bitstream is worked through in Choosing Simulcast vs SVC for Large Conferences.

The cost the stream count hides instead is switching. Every time an SFU promotes a subscriber from the quarter layer to the full layer it must start that subscriber’s decode at a keyframe boundary, which means either waiting for the publisher’s next periodic keyframe or requesting one — and a requested keyframe is 5–10× the size of a delta frame, arriving exactly when the link was already judged good enough to upgrade. An active-speaker layout that reshuffles tiles every few seconds can spend a surprising fraction of its bitrate on switch keyframes alone; damping the churn and hiding the seam is the subject of Switching Layers Without Visible Glitches. An MCU has no equivalent cost because its layout changes happen inside one continuous encode, invisible to the transport.

A hybrid worth naming: many production systems run an SFU as the live topology and bolt an MCU-style compositor on only for server-side recording, so the live call pays SFU economics while the archive gets a single mixed file. That pattern is detailed in Server-Side Recording & Composition.

Step 4 — Verification

Verify the topology behaves as designed by reading getStats() on both the client and the server, polled at 1 s intervals to match the rest of your observability.

// Confirm topology from the subscriber side
const stats = await pc.getStats();
let inboundVideoTracks = 0;
for (const r of stats.values()) {
  if (r.type === 'inbound-rtp' && r.kind === 'video') inboundVideoTracks++;
}
// SFU room of N participants → inboundVideoTracks ≈ N-1 (or your subscribed subset)
// MCU room of any size      → inboundVideoTracks === 1 (the mixed stream)
console.log(`inbound video streams = ${inboundVideoTracks}`);

On the server, confirm an SFU shows near-zero encode time and framesEncoded flat on egress (it is forwarding, not encoding), while an MCU shows totalEncodeTime and decoder utilization climbing with participant count. If an “SFU” reports rising encode time per added participant, something is transcoding when it should be forwarding — a misconfiguration that quietly converts your SFU economics into MCU economics. Making that regression visible over time means shipping totalEncodeTime, forwarded-stream counts, and per-node egress into a dashboard, which is the job described in Exporting SFU Metrics to Prometheus and Grafana. Cross-check the active path and RTT through the candidate-pair report as described in Bandwidth Estimation & Congestion Control.

Named failure modes and how to tell them apart

Three topology-specific failures account for most “the video looks bad and nobody knows why” tickets, and each has a distinct signature in the stats.

Join-time keyframe storm (SFU). Every subscriber that attaches to an existing publisher needs a keyframe before it can render anything, so it sends a PLI. When ten people join a webinar in the same second, the publisher receives ten PLIs and — unless the server coalesces them — encodes several keyframes back to back, briefly tripling its uplink and pushing its own congestion controller into a bitrate cut. The diagnosis is a pliCount on the publisher’s outbound-rtp report that jumps in step with join events, together with keyFramesEncoded rising faster than one per two seconds. The fix is a hold-down in the SFU: absorb duplicate requests inside a window and serve later joiners from a cached keyframe instead of asking the publisher again.

// SFU-side PLI coalescing: one keyframe request per publisher per window
const PLI_WINDOW_MS = 1000;                       // never ask more than 1x/s
function onSubscriberPli(publisher) {
  const now = Date.now();
  if (now - publisher.lastPliSentAt < PLI_WINDOW_MS) return;  // absorb the storm
  publisher.lastPliSentAt = now;
  sendPliUpstream(publisher);                     // single request reaches the encoder
}

Active-speaker flapping (both, worse on MCU). Two people talking over each other, or one participant with a noisy fan, makes the speaker detector oscillate. On an SFU the symptom is subscription churn and the keyframe cost described above; on an MCU it is a layout that visibly jumps every few hundred milliseconds, because the compositor re-renders on every switch. The diagnosis is a speaker-change event rate above roughly one every two seconds in server logs. The fix is hysteresis rather than a better detector: require the challenger’s audio energy to exceed the incumbent’s for 400–600 ms and hold the winner for a minimum 2–3 s before another switch is allowed.

Clock drift in the mix (MCU only). Publishers’ capture clocks are not synchronised, and a webcam that runs at 29.97 fps against an MCU compositor ticking at exactly 30 fps will slowly starve or overfill that input’s buffer. The symptom is one tile in the composite periodically freezing for a frame or duplicating one, on a period of tens of seconds, while every other tile is fine — and it survives a network that shows zero packet loss, which is what rules out congestion. The fix is to resample against the RTCP sender-report timeline rather than counting frames, and to let the compositor drop or repeat a frame deliberately at a chosen boundary rather than letting the buffer do it randomly.

Comparison Table

Dimension SFU MCU
Server media work Route + rewrite RTP headers Decode all + composite + re-encode
Server CPU per participant Very low (no transcode) High (decode + encode per input)
Server downlink (egress) High — up to N-1 streams/participant Low — 1 stream/participant
Client decoders needed Up to N-1 Exactly 1
Mobile / embedded friendliness Limited by decode count Excellent (single decode)
Layout flexibility Client-side, fully flexible Server-fixed per layout
End-to-end latency Lower (no decode/encode hop) Higher (+30–150 ms pipeline)
End-to-end encryption Preserved (payload untouched) Broken (server decrypts + re-encodes)
Codec transcoding / SIP bridge No Yes
Cost driver Bandwidth CPU

The trade-offs in this table are quantified per participant-hour, with concrete CPU and dollar figures, in SFU vs MCU Cost & Quality Trade-offs.

Edge Cases & Browser Quirks

Common Implementation Mistakes

FAQ

Is an SFU always cheaper than an MCU?

It depends which resource you are paying for. An SFU is dramatically cheaper on CPU because it never transcodes, but it is more expensive on egress bandwidth because it emits up to N-1 streams per participant. An MCU inverts that: low egress, high CPU. For most interactive conferencing the SFU’s bandwidth bill is cheaper than the MCU’s compute bill, but a large passive audience watching one fixed layout flips the math toward the MCU.

Can I run both in the same product?

Yes, and many production systems do. Run an SFU for the live, interactive call and invoke MCU-style composition only where a single stream is required — server-side recording, a dial-in phone bridge, or a low-power broadcast endpoint. The live path keeps SFU latency and CPU economics while the composite path gets a single mixed output.

Why does an MCU add latency?

Every MCU output passes through decode → composite/mix → re-encode, a pipeline that adds roughly 30–150 ms depending on codec, resolution, and frame buffering. An SFU only rewrites RTP headers and forwards, so it adds little beyond network transit. For tight conversational latency the SFU wins; for a passive audience the extra delay is usually acceptable.

Does an SFU break end-to-end encryption?

No — that is one of its defining advantages. Because the SFU forwards the encoded payload untouched, it can route media it cannot read, which makes insertable-stream E2EE possible. An MCU must decrypt and decode to composite, so it always sees plaintext and cannot offer true end-to-end encryption.

Can an SFU offer server-controlled layouts like an MCU?

Partly, and the distinction matters. An SFU can control which streams a client receives and can attach layout metadata over a data channel, so the server still decides who is on screen and how large each tile should be. What it cannot do is guarantee the pixels: the client composes the grid itself, so a browser that fails to decode a stream shows a blank tile, and a recording taken from one client’s view is not identical to another’s. If the requirement is “every viewer and the archive see byte-identical frames” — compliance recording, broadcast output, a fixed video wall — that is a mixing requirement, and no amount of SFU-side layout signalling satisfies it.

How many participants can one node carry before topology stops being the limiting factor?

For an SFU the practical ceiling on a single well-provisioned node is usually egress bandwidth and interrupt handling rather than CPU, and it arrives somewhere in the low thousands of forwarded streams — which is a handful of large rooms or a few hundred small ones. For an MCU the ceiling is CPU and it arrives an order of magnitude sooner, typically under a hundred mixed participants per node. Either way, once a single room no longer fits on one machine you have a routing problem rather than a topology problem: rooms have to be pinned to nodes and media cascaded between them, which is why Sharding Rooms Across SFU Nodes becomes the next decision after this one.

Related: start from the Media Server Architecture: SFU & MCU guide, then quantify the decision with SFU vs MCU Cost & Quality Trade-offs, and continue into Selective Forwarding Unit Design, Simulcast-Aware Forwarding, and Load Balancing & Scaling SFUs.

Load Balancing & Scaling SFUs Guide Selective Forwarding Unit Design Guide Server-Side Recording & Composition Guide SFU Observability & Metrics: Instrumenting a Media Server You Can Debug Guide Simulcast-Aware Forwarding Guide Media Server Architecture: SFU & MCU Design, Scaling & Production Patterns Guide