Load Balancing & Scaling SFUs

A single Selective Forwarding Unit process saturates long before your user count does — not on CPU first, but on the outbound network interface, because an SFU’s job is to fan one published stream out to every subscriber in a room. This guide is part of the Media Server Architecture: SFU & MCU guide, and it covers the operational problem that follows once a single node works: how to plan per-node capacity, place rooms onto a pool of nodes, span regions with cascaded relays, drive autoscaling from the right signals, and drain nodes without dropping calls.

The implementation goal here is a horizontally scaled SFU tier that keeps every participant in a room reachable at predictable latency while you add and remove nodes underneath live traffic. The control plane — a load balancer or signaling layer — owns room-to-node assignment; the data plane stays exactly the same per-node forwarding logic described in Selective Forwarding Unit Design. Throughout, assume the per-node forwarding cost model from SFU vs MCU Cost & Quality Trade-offs: an SFU does not transcode, so its cost is bandwidth and packet-forwarding, not CPU-bound composition.

Step 1 — Capacity planning per node by outbound bitrate

Size an SFU node on egress bandwidth first, CPU second. An SFU forwards packets without decoding them, so the dominant constraint is how many bits leave the box. The arithmetic is direct: for a room of N participants where every participant both publishes and subscribes to all others, each published stream is forwarded to N − 1 subscribers, so total egress scales with N × (N − 1) per room. A mesh of small rooms and a few large rooms produce wildly different load on the same hardware.

Work the numbers against a concrete per-stream bitrate. A 720p simulcast stream costs roughly 1.2–1.7 Mbps at its top layer; a node with a 10 Gbps NIC has a hard ceiling near 10,000 Mbps of egress, and you should plan to use no more than 70–80% of it to leave headroom for retransmits and bursts. That puts a single node at roughly 5,000–6,000 forwarded 720p streams before egress saturates, well before a modern multi-core CPU does.

// Estimate whether a node can admit a new room, on outbound bitrate alone.
// Treat egress headroom — not CPU — as the binding constraint for an SFU.
const NIC_MBPS = 10_000;          // 10 Gbps interface
const SAFE_FRACTION = 0.75;       // leave 25% for retransmits, RTX, bursts
const MAX_EGRESS = NIC_MBPS * SAFE_FRACTION;

// Per-room egress: every publisher is forwarded to every other subscriber.
function roomEgressMbps(participants, perStreamMbps = 1.5) {
  return participants * (participants - 1) * perStreamMbps; // N*(N-1) fan-out
}

function canAdmit(node, participants, perStreamMbps = 1.5) {
  const projected = node.currentEgressMbps + roomEgressMbps(participants, perStreamMbps);
  return projected <= MAX_EGRESS; // reject if it would breach the safe ceiling
}

// A 50-person all-to-all room costs 50*49*1.5 ≈ 3,675 Mbps — over a third of one node.

Track currentEgressMbps from each node’s own forwarding stats (sum of outbound-rtp bytesSent deltas over a 1 s window, the same getStats cadence used everywhere else in this stack) rather than from a static per-participant estimate, because simulcast layer selection means actual egress varies with what subscribers can receive — the mechanism detailed in Bandwidth-Aware Layer Selection in an SFU. Publish that per-node egress gauge as a scrape target rather than keeping it internal to the admission check, following Exporting SFU Metrics to Prometheus and Grafana, so the same number drives placement, dashboards and the autoscaler.

Why the interface saturates before the cores do

An SFU never decodes a frame, but it does not merely copy bytes either: every forwarded copy is re-encrypted. Each subscriber negotiates its own DTLS-SRTP session, so one publisher packet destined for 49 subscribers is encrypted 49 times, under 49 different keys with 49 independent rollover counters. That sounds expensive until you price it — AES-128-GCM runs at roughly 1–2 GB/s per core with AES-NI, and 7,500 Mbps of egress is about 940 MB/s, well under a single core’s cipher throughput. This is exactly why headline CPU utilisation on a busy SFU stays misleadingly low and why a CPU-triggered autoscaler is blind.

The cost that actually bites is per-packet, not per-byte. A 1.5 Mbps video stream with ~1,100-byte payloads is roughly 170 packets per second, so 5,000 forwarded streams means about 850,000 packets per second leaving the box. At 1.5–3 µs of kernel send path per sendmsg, a single-threaded forwarding loop runs out of headroom somewhere near 400–600 k pps — long before the NIC’s line rate. Two fixes matter more than all others: batch with sendmmsg() so one syscall carries 32–64 datagrams, and shard sockets across several sender threads pinned to distinct NIC transmit queues. The symptom of getting this wrong is latency, not loss: the 1–3 ms an SFU normally adds per hop is a queueing figure, and once softirq backlog builds it climbs into the tens of milliseconds while the drop counters are still at zero.

# Kernel tuning for a high-fan-out SFU node — per-packet cost is the real ceiling.
net.core.wmem_max = 33554432          # 32 MB send-buffer ceiling for UDP sockets
net.core.rmem_max = 33554432          # matching receive ceiling for publisher ingress
net.core.netdev_max_backlog = 30000   # queue depth before softirq lag causes drops
net.ipv4.udp_mem = 786432 1048576 1572864   # pages: pressure / low / high for UDP
# Then give the box one RX/TX queue per core and pin sender threads to match:
# ethtool -L eth0 combined 16

The 25% reserve in SAFE_FRACTION is not superstition either. NACK-driven RTX adds 5–15% on top of the media rate during a loss episode, and every keyframe is 3–5× the size of a P-frame. When a 50-person room takes a network hiccup and thirty subscribers request a keyframe inside the same second, instantaneous egress spikes far above the steady-state number the admission check measured.

Step 2 — Room-to-node assignment

The signaling layer, not the SFU, decides which node a room lives on. The default and correct policy for most workloads is room affinity: every participant in one room connects to the same node, so the SFU can forward locally without any cross-node hop. The load balancer’s only job at join time is to pick a node for the first participant of a room and then pin every subsequent joiner to that same node.

Pick the node by least projected egress, not round-robin and not least-connections. Round-robin ignores that one 50-person room outweighs fifty 2-person rooms; least-connections has the same blindness. A capacity-aware “least loaded by egress headroom” assignment keeps nodes evenly filled in the dimension that actually saturates.

// Room-affinity assignment: pin a room to one node, chosen by egress headroom.
// Backed by a shared store so every signaling instance agrees on placement.
async function assignRoomToNode(roomId, expectedParticipants, store, nodes) {
  const existing = await store.get(`room:${roomId}:node`);
  if (existing) return existing; // affinity — never split a room across nodes

  const projected = roomEgressMbps(expectedParticipants);
  const candidates = nodes
    .filter(n => n.healthy && !n.draining)                  // skip draining nodes
    .filter(n => n.currentEgressMbps + projected <= MAX_EGRESS)
    .sort((a, b) => a.currentEgressMbps - b.currentEgressMbps); // least loaded first

  if (candidates.length === 0) throw new Error('NO_CAPACITY'); // trigger scale-up
  const chosen = candidates[0].id;

  // Claim atomically so concurrent first-joins don't race onto two nodes.
  await store.setNX(`room:${roomId}:node`, chosen, { ttl: 3600 });
  return await store.get(`room:${roomId}:node`);
}

The shared store that holds room → node mappings is the same Redis instance most teams already run for Scaling WebSocket Signaling with Redis Pub/Sub, so signaling fan-out and room placement stay consistent across every signaling node. The consistent-hashing and failover details of this mapping are the subject of Sharding Rooms Across SFU Nodes.

Admitting on estimates, then correcting for drift

assignRoomToNode places a room on a guess. expectedParticipants comes from a calendar invite, a room-type default, or nothing at all, while the room’s real egress ramps over the 30–60 s it takes people to actually join. Two distinct failure modes follow from that gap.

Estimate drift. A room booked for 10 participants grows to 30. Its egress goes from 135 Mbps to 1,305 Mbps — nearly ten times, because the fan-out is quadratic while the booking error was only linear — and the node that accepted it is now silently over-committed. Diagnose it by charting each node’s summed projected egress against its measured egress; a node whose measured line crosses above its projected line is carrying rooms nobody budgeted for. The fix is a two-level water mark: refuse new room assignments at 65% of the safe ceiling, but let existing rooms keep growing until 80%, so a room that outgrows its estimate is never the thing that gets rejected.

Placement races on a stale gauge. The egress gauge is a 1 s sample, and a freshly assigned room contributes nothing to it for several seconds. Three signaling instances placing three large rooms inside the same window will all read the same emptiest node and all pick it. Fix it with reservations: write the projected egress alongside the room:{id}:node claim, have the placement filter sum outstanding reservations plus measured egress, and decay each reservation as the room’s real numbers arrive. Without it, the setNX claim protects one room from splitting but does nothing to stop three rooms from colliding onto one node.

Very small rooms raise the opposite question. A 4-person call costs 12 forwarded streams — 18 Mbps, a rounding error — yet still consumes a DTLS session per peer and a slot in every dashboard, so it may not belong on the SFU tier at all; the crossover point is worked through in Mesh vs SFU: When to Graduate.

Step 3 — Cascaded SFUs for cross-region and oversized rooms

Affinity breaks in two cases: a room larger than one node can hold, and a room whose participants are spread across regions. Both are solved the same way — cascading, where two SFU nodes subscribe to each other and relay a room’s streams over a single inter-node link instead of forwarding to every remote participant directly.

For cross-region rooms, pin participants to the SFU in their own region and cascade only the aggregate room media between the regional nodes. A participant in Frankfurt subscribes to the EU node; that EU node holds one relayed copy of each US publisher pulled across the Atlantic once, rather than each US publisher’s stream crossing the ocean per EU subscriber. This collapses inter-region egress from publishers × remote_subscribers to publishers × regions, and keeps each participant’s first hop on a low-latency regional path — the same multi-region latency win (40–60% on connect) that regional STUN placement buys at the ICE layer.

// Cascade: a local node pulls each remote publisher exactly once from the peer node.
// One relayed copy per publisher per region — not per remote subscriber.
async function ensureCascade(localNode, remoteNode, roomId, publishers) {
  for (const pub of publishers) {
    const key = `cascade:${roomId}:${pub.id}:${localNode.id}`;
    if (localNode.hasRelay(key)) continue;     // already pulling this publisher

    // localNode acts as a subscriber to remoteNode for this publisher's stream,
    // then re-forwards it to its own local subscribers as if locally published.
    const relay = await localNode.subscribeRemote(remoteNode, pub.id);
    localNode.registerRelay(key, relay);
  }
  // Forward your local publishers to remoteNode symmetrically (it pulls from you).
}

Cascading is not free: it adds one relay hop of latency (typically 80–150 ms inter-region) and doubles the bookkeeping. Reserve it for rooms that genuinely span regions or exceed single-node capacity; keep ordinary rooms node-local. The same simulcast-aware forwarding rules apply on the relayed copy — the cascade should pull only the layers some remote subscriber actually needs, as covered in Simulcast-Aware Forwarding.

Cascade topology and loop prevention

With two regions there is one link and no decision to make. With three or more you choose between a full mesh of cascades — every regional node pulls from every other, costing publishers × (R − 1) relayed copies — and a star through a hub region, which costs one relayed copy per spoke but forces spoke-to-spoke media through two relay hops. Two hops at 80–150 ms each puts spoke-to-spoke mouth-to-ear latency at 160–300 ms, past the point where conversational turn-taking degrades. Stay with a full mesh up to four or five regions: the extra copies are cheap next to the latency a hub imposes, and a mesh degrades gracefully when one region’s node dies.

A mesh cascade needs explicit loop prevention. Cascade echo is the failure mode: node A relays publisher P to node B, node B treats the relayed copy as an ordinary local publisher and offers it onward to node C, and node C relays it back to A, which no longer recognises its own stream and fans it out again. The signature is unmistakable once you know it — egress climbs steadily while participant count stays flat, the same SSRC appears twice in one room’s forwarding table, and subscribers see the speaker’s video duplicated in the roster. The fix is a one-line invariant: tag every stream with the id of the node that originally received it from a browser, and refuse to relay any stream whose origin tag is not your own. Relays are strictly one hop, always.

Keyframe amplification is the second cascade-specific hazard. Every new remote subscriber needs a decodable starting point, so a join on the EU node sends a PLI back across the ocean to the original publisher; ten EU joins in a burst produce ten full keyframes at 3–5× a P-frame each, on a link you deliberately sized for one copy per publisher. Coalesce PLIs at the relay edge into at most one upstream request per publisher per second and serve the intervening subscribers from a cached keyframe — the debouncing and cache strategies in Keyframe Request Strategies in an SFU apply unchanged at the cascade boundary.

Step 4 — Verification: autoscaling signals and draining nodes

Verify the tier the way it will behave under real load: scale up before saturation, scale down without dropping calls, and confirm both with the node-level egress signal you already collect.

Autoscaling signals. Scale on aggregate egress utilization and projected admission failures, not on CPU. CPU on an SFU stays low until egress is long gone, so a CPU-based autoscaler reacts far too late. Trigger scale-up when the pool’s mean egress crosses ~65% of the safe ceiling for 60 s, or when any assignRoomToNode returns NO_CAPACITY. Because new rooms can pin to a fresh node immediately but existing rooms cannot migrate without a renegotiation, scale up earlier than you would a stateless web tier; the concrete thresholds, cooldown windows and why a CPU-only policy misfires are worked through in Autoscaling SFU Nodes on CPU and Bandwidth.

Draining nodes. A node never hard-stops while it holds live rooms. Mark it draining so the load balancer stops assigning new rooms to it, let existing rooms finish naturally, and only terminate once participant count reaches zero — or, for long-lived rooms, migrate them by signaling affected clients to reconnect, which re-runs assignment onto a healthy node.

// Drain a node: stop new assignments, wait for rooms to empty, then terminate.
async function drainNode(nodeId, store, opts = { maxWaitMs: 1_800_000 }) {
  await store.set(`node:${nodeId}:draining`, '1');   // load balancer skips it now
  const start = Date.now();

  while (Date.now() - start < opts.maxWaitMs) {
    const rooms = await store.smembers(`node:${nodeId}:rooms`);
    if (rooms.length === 0) break;                   // empty — safe to terminate

    // For rooms that won't drain on their own, ask clients to reconnect so the
    // load balancer reassigns them to a healthy node (brief ICE re-handshake).
    if (Date.now() - start > opts.maxWaitMs / 2) {
      for (const roomId of rooms) await signalReconnect(roomId);
    }
    await sleep(5_000);                              // re-poll, don't spin
  }
  await terminateNode(nodeId); // scale-in only after the node is empty
}

The reconnect-driven migration is a normal ICE restart from the client’s view — createOffer({ iceRestart: true }) against the new node — so cap it at 3 attempts with a 3–5 s fallback, exactly as the Signaling State Machine Patterns guide bounds every reconnection path, and keep the media flowing across the swap using the technique in Triggering an ICE Restart Without Dropping Media. Confirm a clean drain by watching the node’s egress fall to zero before scale-in fires, and spot-check one migrated participant end to end — Tracing One Participant Across SFU Logs shows how to follow a single session as it moves from the drained node to its replacement.

Rehearsing a drain under synthetic load

A drain path that has only ever run during an incident is not a drain path. Rehearse it on the same cadence as your deploys, with synthetic rooms sized like your real p99 room, and check four things in order:

1. Load one node to roughly 60% of its safe ceiling and record the steady-state egress gauge — this is the baseline the drain has to walk back to zero.
2. Set the draining flag. The correct signature is a room set that stops growing while egress stays flat; egress that drops immediately means you tore sessions down rather than stopping assignment.
3. At half the window, confirm migration fires for the stragglers and that reconnects are jittered. A synchronised herd shows up as a spike in signaling message rate and a block of near-identical join timestamps; jittered reconnects spread the same joins over 1–5 s.
4. Confirm each migrated participant completes ICE against the new node within 200–800 ms of its first trickled candidate, and that its first outbound-rtp sample on the replacement appears inside 3–5 s. Slower than that means the reconnect fell through to the fallback timeout, and the retry budget should stay capped at 3.

Only call the drain clean when the egress gauge reads zero across two consecutive 1 s samples and the room set is empty. Freeze and packet-loss alerts should stay silent throughout; if they fire during a rehearsal, the migration path is dropping media rather than moving it.

Edge Cases & Browser Quirks

• Reconnect storms on scale-in. Draining a node by mass-reconnect can stampede the signaling layer if every client retries at once. Jitter reconnect delays across a 1–5 s window per client; Chrome and Firefox both honor an immediate iceRestart offer, so without jitter you get a synchronized thundering herd onto the replacement node.
• Safari renegotiation on migration. Safari (WebKit) is stricter about m-line ordering on the post-migration offer than Chrome. A room migrated to a new node must reproduce the original transceiver order or Safari rejects the answer — see Debugging SDP m-line Mismatches. Pin transceiver order server-side rather than letting the new node re-derive it.
• Cascaded RTCP feedback. PLI and NACK feedback must traverse the cascade hop, adding 80–150 ms to keyframe recovery for remote subscribers. Firefox is more aggressive about requesting full keyframes on packet loss than Chrome, so a cross-region cascade shows more inter-region keyframe traffic when EU subscribers run Firefox.
• CGNAT participants behind a relayed first hop. When a participant is already on a TURN relay (symmetric NAT / CGNAT), their first hop to the regional SFU is itself relayed; binding refreshes under 30 s still apply, and a node drain must not outlive the TURN allocation lifetime or the migrated session dies silently.
• Sticky load-balancer hashing vs WebSocket upgrade. An L4 load balancer that re-hashes on reconnect can land a returning client on a signaling node that doesn’t hold its room mapping. Always resolve placement from the shared store, never from local in-memory state on the signaling node.
• Bandwidth estimate reset on migration. Chromium restarts its congestion controller from the configured start bitrate after an ICE restart onto a new node, so a migrated participant spends 2–5 s ramping back up instead of resuming at its previous rate, and viewers read that as a quality dip caused by your maintenance window. For reconnects you initiated yourself you already know the pre-migration rate from the old node’s outbound-rtp samples, so raise the start bitrate rather than letting the estimator rediscover it.
• Clock skew between cascaded nodes. Lip sync is reconstructed from RTCP sender reports mapping RTP timestamps to wall clock. A relay that regenerates sender reports from its own clock injects that node’s skew into the stream: 100 ms of drift between two regional nodes is audible A/V offset for every remote subscriber, and it is baked permanently into server-side recordings. Keep cascade nodes on one NTP source and forward the original mapping instead of recomputing it.
• Idle cascade links torn down by stateful firewalls. An inter-node relay for a room where everyone is muted and camera-off carries no media, and a stateful firewall or cloud NAT will reclaim the mapping. Keep an explicit keepalive inside the same sub-30 s budget you use for STUN binding refreshes, or the first person to unmute after a long silence is inaudible for several seconds while the link re-establishes.

Common Implementation Mistakes

• Splitting one room across nodes by default. Spreading a room’s participants over multiple nodes for “balance” forces a cascade hop on every internal subscription and multiplies inter-node bandwidth. Keep rooms node-local; cascade only when a room is too big for one node or genuinely multi-region.
• Balancing on connection count or CPU. Both ignore egress, the dimension that actually saturates an SFU. A node with few but huge rooms looks idle by connection count and by CPU right up until its NIC is full. Assign and autoscale on projected egress.
• Cascading per-subscriber instead of per-publisher. Pulling a remote publisher once per remote subscriber defeats the entire purpose of cascading. Pull each remote publisher exactly once per node and re-forward locally.
• Hard-killing nodes on scale-in. Terminating a node that still holds rooms drops every call on it. Always drain — stop new assignments, wait, then migrate stragglers via client reconnect.
• Placing room state only in node memory. If the room → node map lives only on the assigning signaling instance, any other instance and any failover loses it. Persist it in Redis so placement survives node failure and rebalance, as detailed in the sharding guide.
• Forwarding all simulcast layers across the cascade. Relaying every layer between regions wastes the inter-region link. Pull only layers a remote subscriber currently needs.
• Feeding the autoscaler the same raw gauge you admit on. Admission needs the instantaneous 1 s egress sample; scaling decisions need a 60 s smoothed one. Wire the raw value into both and the pool oscillates — a keyframe burst trips scale-up, the burst passes, scale-in fires, and you pay a node’s boot time on every hiccup.
• Leaving draining nodes in the pool utilisation average. A node that is deliberately emptying drags the pool mean down and can suppress the scale-up that the remaining nodes need. Exclude draining nodes from both the numerator and the denominator of the utilisation metric the autoscaler reads.
• Sizing the pool on mean room size. The mean is dominated by two-person calls while the capacity is dominated by the tail, because egress goes with N × (N − 1). Provision against your p99 room size and the number of such rooms you expect concurrently, not against average concurrency.

FAQ

How many participants can one SFU node handle? On egress, not CPU. With a 10 Gbps NIC at 75% safe utilization and ~1.5 Mbps per forwarded 720p stream, a node tops out near 5,000–6,000 simultaneous forwarded streams — which is a few large rooms or many small ones. Compute the ceiling for your own per-stream bitrate with N × (N − 1) egress per room rather than a flat per-user number.

When should I cascade rooms across nodes instead of keeping them on one? Only when a room exceeds a single node’s egress ceiling or its participants are split across regions. Cascading adds an 80–150 ms relay hop and roughly doubles bookkeeping, so node-local affinity is the default; reach for a cascade as the exception, and pull each remote publisher exactly once per node.

What signal should drive SFU autoscaling? Aggregate egress utilization across the pool plus admission-failure (NO_CAPACITY) events — never CPU, which stays low until egress is already exhausted. Scale up at ~65% mean egress for 60 s and scale up earlier than a stateless tier, because existing rooms can’t migrate to a new node without a client renegotiation.

How do I remove a node without dropping calls? Drain it: flag it so the load balancer stops assigning new rooms, let existing rooms empty naturally, and for long-lived rooms migrate participants by signaling a reconnect that re-runs assignment onto a healthy node. Only terminate once the node’s egress reaches zero.

How wide does the inter-region cascade link need to be? Size it as publishers × layers_pulled × per_layer_bitrate × (regions − 1), never against the remote subscriber count. Twenty active publishers, two simulcast layers pulled per publisher, and roughly 1.9 Mbps for those two layers combined comes to about 38 Mbps per region pair — small enough that the link is almost never the constraint once you cascade per publisher instead of per subscriber. Budget the audio separately at 24–32 kbps per publisher; it is negligible in bandwidth but it must never be dropped from the relay, because a subscriber with video and no audio is a worse outage than the reverse.

Should cascade traffic run over TURN? No. TURN exists to get a browser through a NAT it does not control; on top of the 80–150 ms the ocean already costs it adds 20–40 ms one-way and a relay allocation you have to keep alive. Your SFU nodes have routable addresses and a port range you own, so terminate cascades directly with mutual authentication between nodes. If two nodes genuinely cannot reach each other, fix the network peering rather than pushing relayed media through a TURN server sized for browsers.

Related: start from the Media Server Architecture: SFU & MCU guide, then pair this with Sharding Rooms Across SFU Nodes for the room-affinity map, SFU vs MCU Cost & Quality Trade-offs for the per-node cost model, and Selective Forwarding Unit Design for the forwarding logic each node runs.