Autoscaling SFU Nodes on CPU and Bandwidth

An autoscaler that watches CPU will add capacity to a Selective Forwarding Unit tier several minutes after the first participant has already started freezing. This guide is part of the Load Balancing & Scaling SFUs guide, and it settles one operational question: which numbers should drive desiredCount on a tier whose nodes hold live, unmovable sessions, and how do you remove a node from that tier without cutting a call. The answer is a composite load score built from outbound bitrate and forwarded-stream count, with CPU demoted to a guardrail, plus a two-phase removal — cordon, then drain.

Context & Trade-offs

An SFU relays packets; it does not decode or re-encode them. Forwarding a stream costs roughly 1–3 ms of packet handling versus the 80–200 ms of transcode work an MCU spends per composed output, which is exactly why the CPU number is uninformative. On a 2 vCPU node with a 1 Gbps interface, SRTP encryption and the NACK retransmission cache typically hold CPU between 20% and 30% while the interface is already pushing 900 Mbps. The box is finished; the metric your cloud provider hands you by default says it is idle.

Two signals actually track exhaustion. The first is outbound bitrate — the sum of bytesSent deltas across every outbound-rtp the node maintains, sampled on the same 1 s cadence used everywhere else in this stack. It must be measured, not estimated from participant counts, because what a node really sends depends on which simulcast layer each subscriber is receiving, the selection logic covered in Bandwidth-Aware Layer Selection in an SFU. The second is forwarded-stream count, because a large share of per-node work is fixed per stream rather than per byte: header rewriting, sequence-number continuity, one SRTP context, one jitter-buffer-adjacent bookkeeping structure. A node carrying 900 audio-only streams at 32 kbps each occupies barely 30 Mbps but has 900 sets of per-stream state, and it will fall over on stream count long before bitrate looks interesting.

Both ceilings have to be measured on your own hardware before they mean anything. Set the egress ceiling at roughly 70% of the interface rate — 700 Mbps on a 1 Gbps NIC — because the remaining headroom absorbs keyframe bursts, and a keyframe storm after a layer switch can add 30–40% to instantaneous egress for a second or two. The stream ceiling comes from a load test rather than arithmetic: push forwarded streams up in steps of 100 until the node’s own forwarding delay leaves the 1–3 ms band, and take 80% of the count where it broke. A node with a 700 Mbps egress ceiling and a 1,200-stream ceiling behaves completely differently in a 40-person video meeting (bitrate-bound) than in a 600-seat webinar with audience audio (stream-bound), and the composite score is what lets one policy serve both.

Take the max of the normalised signals rather than an average: an average of a 0.92 egress ratio and a 0.15 stream ratio reads as a comfortable 0.53 and never scales. The remaining trade-off is timing, and it is specific to stateful media. A new node absorbs no existing load — placement is sticky, as described in Sharding Rooms Across SFU Nodes — so a fresh node only relieves the pool at the rate new rooms are created. Scale out at 0.65, not 0.85, and accept that you are paying for headroom you will not use for several minutes.

Minimal Runnable Implementation

The control loop below runs on one leader instance, ticks every 30 s, and produces at most one pool change per cooldown window.

const TICK_MS = 30_000;        // one control tick every 30 s
const UP_AT = 0.65;            // pool mean load that triggers scale-out
const DOWN_AT = 0.30;          // pool mean load that permits scale-in
const UP_STREAK = 4;           // 4 of the last 5 ticks ≈ 2 min of evidence
const DOWN_STREAK = 20;        // 10 min of quiet before shrinking
const COOLDOWN_MS = 300_000;   // 5 min between any two pool changes
const FLOOR = 3;               // never shrink past 3 nodes

// One node's load is the max of its real constraints. CPU is a guardrail:
// divided by 0.85 it only becomes the max once encryption work truly dominates.
function nodeLoad(n) {
  const egress = n.egressMbps / n.egressCeilingMbps;    // 1 s bytesSent deltas
  const streams = n.forwardedStreams / n.streamCeiling; // fixed per-stream cost
  const cpuGuard = n.cpu01 / 0.85;                      // 0.85 CPU => 1.0 load
  return Math.max(egress, streams, cpuGuard);
}

let upHits = [], downHits = [], lastChangeAt = 0;

async function tick(pool, store) {
  const live = pool.filter(n => n.healthy && !n.cordoned); // cordoned ≠ capacity
  const mean = live.reduce((s, n) => s + nodeLoad(n), 0) / live.length;

  upHits = [...upHits, mean >= UP_AT].slice(-5);          // rolling 5-tick window
  downHits = [...downHits, mean <= DOWN_AT].slice(-DOWN_STREAK);
  if (Date.now() - lastChangeAt < COOLDOWN_MS) return;    // one change per window

  if (upHits.filter(Boolean).length >= UP_STREAK) {
    await pool.launchNode();                              // absorbs NEW rooms only
    lastChangeAt = Date.now();
    upHits = [];                                          // reset both streaks
    return;
  }

  if (downHits.length === DOWN_STREAK && downHits.every(Boolean) &&
      live.length > FLOOR) {
    // Pick the node cheapest to empty, not the emptiest by CPU.
    const victim = [...live].sort((a, b) => a.forwardedStreams - b.forwardedStreams)[0];
    await store.set(`node:${victim.id}:cordoned`, '1', 'EX', 3600); // step 1: cordon
    victim.drainDeadline = Date.now() + 1_800_000;         // step 2: 30 min drain
    lastChangeAt = Date.now();
    downHits = [];
  }
}

setInterval(() => tick(pool, store).catch(console.error), TICK_MS);

Three details in that loop are easy to get wrong. The streak arrays are the hysteresis: without them a single 30 s tick containing one large room’s join burst launches a node you pay for and never fill. The loop launches exactly one node per decision rather than a proportional jump, because the pool’s mean is measured over nodes that are already saturated and cannot shed load — adding three nodes at once does not relieve the existing rooms three times faster, it just moves the next four minutes of new rooms onto cold hardware. And the loop must have exactly one writer; run it behind a leader lock in the same store that holds placement state, or two instances will each decide the pool is short and launch a node in the same tick.

Cordoning is the part that makes scale-in safe, and it is deliberately not the same thing as a failed health check. A cordoned node is healthy, keeps forwarding every packet for the rooms it already owns, and simply stops being a placement candidate — the flag lives in the same shared store the room-to-node mapping uses, so every signaling instance sees it on the next join without a deploy or a load-balancer change. Do not implement cordon by pulling the node out of service discovery: that would strand the participants already on it. Publish the cordon flag and the drain deadline as gauges alongside your load signals, following Exporting SFU Metrics to Prometheus and Grafana, or you will spend the drain window unable to tell a stuck node from a busy one.

Reproduction Steps & Debugging Log Patterns

1. Write the policy the loop reads at boot, sizing the ceilings below the hardware rather than at it.

# sfu-autoscaler.ini — ceilings are per node, not per pool
[signals]
egress_ceiling_mbps = 700     ; 70% of a 1 Gbps NIC, leaves burst headroom
stream_ceiling      = 1200    ; forwarded outbound-rtp streams per node
cpu_guard           = 0.85    ; CPU only counts once it passes this
[policy]
scale_out_at = 0.65           ; pool mean load, sustained
scale_in_at  = 0.30
cooldown_s   = 300            ; no two pool changes inside 5 minutes
drain_max_s  = 1800           ; then stragglers are asked to reconnect
min_nodes    = 3

2. Ramp synthetic rooms onto a 4-node pool until mean load passes 0.65 and confirm the streak logic waits out the noise instead of firing on a single spike.

// [scale] tick mean=0.61 egress=0.61 streams=0.22 cpu=0.28  hits=1/5
// [scale] tick mean=0.67 egress=0.67 streams=0.24 cpu=0.29  hits=2/5
// [scale] tick mean=0.66 egress=0.66 streams=0.24 cpu=0.29  hits=3/5
// [scale] tick mean=0.69 egress=0.69 streams=0.25 cpu=0.30  hits=4/5 -> LAUNCH
// [scale] node-e launching  pool=4->5  cooldown_until=+300s

3. Read the same tick with only the CPU column. It never leaves the 0.28–0.30 band, which is the whole argument for the composite score.
4. Force the opposite direction by ending rooms until mean load sits under 0.30 for 20 consecutive ticks, then watch the cordon land before anything terminates.

// [scale] mean=0.27 for 20 ticks, pool=5 > floor=3 -> CORDON node-e
// [place] node-e cordoned: skipped for room r-5512 (placed node-b)
// [drain] node-e streams=143 rooms=11 deadline=+1800s
// [drain] node-e streams=0 rooms=0 elapsed=944s -> TERMINATE

5. Verify nothing was cut: across the drain window the node’s connectionState=failed count must stay at zero, and its egress must reach zero before termination, not at it. A single failed here means you terminated on the deadline while rooms were still live.

The one measurement people skip is warm-up, and it is the difference between a scale-out that helps and one that produces a wave of failed joins. Time the interval between the launch call and the new node’s first successful forwarded packet: on a typical image that is 20–45 s of boot, certificate load and port binding, and the node will answer a health check well before the end of it. Log the gap explicitly and refuse to count the node as capacity until it closes, otherwise the placement layer hands rooms to a node that cannot yet complete DTLS.

// [pool] node-e launch requested            t+0s
// [pool] node-e health=ok streams=0         t+11s  (NOT capacity yet)
// [pool] node-e first forwarded packet      t+34s  -> counted as capacity
// [place] room r-5610 -> node-e             t+36s  (first room lands)

Common Implementation Mistakes

• Averaging the signals instead of taking the max. A node at 0.92 egress and 0.15 streams averages to 0.53 and reads as healthy. The constraint that binds is whichever ratio is highest, so the score must be a max.
• Scaling on the same threshold in both directions. Scale-out at 0.65 and scale-in at 0.65 makes the pool oscillate: adding a node drops the mean below the line, which immediately justifies removing it. Keep a wide gap — 0.65 up, 0.30 down — and require a much longer streak downward.
• Cordoning by removing the node from service discovery. That is a hard kill wearing a cordon’s name. Cordon must be a placement-only flag that leaves existing forwarding and the node’s health endpoint untouched.
• Terminating on the drain deadline regardless of occupancy. A 30-minute deadline is when you start asking the remaining clients to reconnect, not when you power off. Migrate stragglers with a bounded, jittered renegotiation as described in Triggering an ICE Restart Without Dropping Media, then terminate at zero streams.
• Using instantaneous samples. A single 1 s getStats window catches keyframe bursts and reads 30–40% above steady state. Smooth over the 30 s tick before comparing to any threshold.

FAQ

Should CPU be in the policy at all?

Yes, but only as a ceiling that rarely binds. Divide measured CPU by 0.85 before folding it into the max, so it contributes nothing until the node is genuinely spending its cores on SRTP and NACK retransmission — which happens on tiers with very many small audio streams, and almost never on video-heavy ones.

How long should the cooldown be?

Five minutes between pool changes, and it must cover node boot plus warm-up. An SFU node that has joined service discovery but has not yet loaded its certificates or opened its RTP ports will accept a room and fail it; count the node as capacity only after it reports its first successful forwarding tick.

Why not scale in faster when the pool is obviously idle?

Because idle is measured on bitrate, and a room can go from idle to full-rate the moment someone unmutes a camera. A 10-minute downward streak plus a 30-minute drain is cheap insurance against removing the node that was about to be needed; if you want the freeze and loss consequences of getting it wrong to be visible, wire the pool into the thresholds from Alerting on Freeze and Packet-Loss SLOs.

Related: this sits under Load Balancing & Scaling SFUs; read it with SFU vs MCU Cost & Quality Trade-offs for the per-node cost model that sets your ceilings.