Rewriting RTP Header Extensions When Forwarding

A forwarder that copies a publisher’s RTP header extension block byte-for-byte onto a subscriber’s transport will silently poison that subscriber’s bandwidth estimator, confuse its demultiplexer, and — the first time it switches simulcast layers — hand the jitter buffer a sequence discontinuity it never recovers from. This guide is part of the Selective Forwarding Unit Design guide, and it settles one question field by field: for every value in the RTP header and the extension block behind it, does the forwarder copy it, remap its ID, recompute it from scratch, or drop it entirely?

Context & Trade-offs

The trap is that extension IDs are not global. RFC 8285 packs extensions into a block introduced by the profile word 0xBEDE, where each element carries a 4-bit local ID and a 4-bit length — and that ID is negotiated independently on every RTCPeerConnection. Chrome, Firefox and Safari each assign a=extmap numbers in their own order, and the numbering the publisher agreed with the server has no relationship to the numbering a subscriber agreed. A forwarder is therefore a translator, not a relay: it decodes ingress ID to canonical URI, then encodes canonical URI to the egress ID that particular subscriber negotiated.

; Ingress: what the publisher's browser happened to pick in its answer
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
; Egress: a different subscriber, different browser, different numbers for the same URIs
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:7 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
; no rtp-stream-id on egress at all — the subscriber receives one already-selected stream
a=extmap-allow-mixed

Beyond renumbering, the values themselves fall into three classes. Some are owned by the sending hop and must be regenerated: abs-send-time encodes when this transport put the packet on the wire, so forwarding the publisher’s stamp adds the entire first-hop queueing and transit delay into the subscriber’s inter-arrival deltas, and the estimator reads that constant offset as jitter — typically costing several hundred kbps of usable estimate on a relayed path that already carries 20–40 ms of TURN latency. The transport-wide sequence number is worse: it indexes the transport, not the stream, so two forwarded publishers copied verbatim will emit duplicate and non-monotonic counters and the subscriber’s feedback becomes unparseable. Why that counter matters at all, versus the coarser aggregate alternative, is worked through in Transport-CC vs REMB Feedback.

Some are identity extensions. mid must be rewritten to the mid of the subscriber’s own m-line, because that is how the receiving browser routes the packet to the right transceiver; ship the publisher’s mid and the packet lands on the wrong RTCRtpReceiver or nowhere. rid and repaired-rid should normally be stripped, since the whole point of the forwarder is that the subscriber sees one stream whose layer identity the server has already chosen on its behalf. The rest — video-orientation, playout-delay, the AV1 dependency descriptor — are opaque: remap the ID, copy the bytes, never interpret them.

None of this is expensive. Decoding a one-byte extension block, mutating three or four elements and re-serialising costs a few microseconds per packet and keeps total forwarding overhead in the familiar 1–3 ms range, against 80–200 ms for a transcoding topology. The cost of getting it wrong is not CPU, it is a subscriber whose estimate collapses for reasons no getStats() dashboard explains.

Minimal Runnable Implementation

// Canonical URIs are the only stable key between the two negotiated ID spaces.
const EXT = {
  ABS_SEND_TIME: 'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
  TWCC: 'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01',
  MID: 'urn:ietf:params:rtp-hdrext:sdes:mid',
  RID: 'urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id'
};

// abs-send-time is 24 bits of 6.18 fixed-point seconds, so it wraps every 64 s.
function absSendTime24(nowMs) {
  return Math.floor((nowMs / 1000) * 262144) & 0x00ffffff; // 2^18 ticks per second
}

// Returns the egress extension list; caller re-serialises and pads to a 4-byte boundary.
function rewriteExtensions(pkt, sub, nowMs) {
  const out = [];
  for (const el of pkt.extensions) {             // parsed from the publisher's 0xBEDE block
    const uri = pkt.ingressExtmap.get(el.id);    // publisher's local id -> URI
    const outId = uri && sub.egressExtmap.get(uri); // URI -> this subscriber's local id
    if (!outId) continue;                        // not negotiated on egress: drop, never guess

    switch (uri) {
      case EXT.ABS_SEND_TIME:
        // Means "when THIS hop sent it". Forwarding the publisher's stamp injects hop-1 delay.
        out.push({ id: outId, bytes: u24(absSendTime24(nowMs)) });
        break;
      case EXT.TWCC:
        // Owned by the egress transport, shared across every stream on it. 16-bit wrap.
        sub.twccSeq = (sub.twccSeq + 1) & 0xffff;
        out.push({ id: outId, bytes: u16(sub.twccSeq) });
        break;
      case EXT.MID:
        // The subscriber's own m-line mid: ASCII, no terminator, length lives in the header.
        out.push({ id: outId, bytes: ascii(sub.mid) });
        break;
      case EXT.RID:
        break;                                   // one selected stream out: rid has no meaning
      default:
        out.push({ id: outId, bytes: el.bytes }); // opaque payload: remap id, copy bytes
    }
  }
  return out;
}

Two serialisation details bite in production. A one-byte element can carry 1–16 data bytes and an ID of 1–14, so a mid longer than 16 characters or an egress ID above 14 forces the two-byte form (0x1000) — only legal if both sides negotiated a=extmap-allow-mixed, which Safari did not send before 15.4. Keep server-assigned mids short (v0, a1) and egress IDs low and this never arises. Second, the extension block length is measured in 32-bit words, so after dropping rid you must recompute the length and re-pad with zero bytes; leaving the old length is the single most common cause of a receiver that reports every packet as malformed while packetsReceived still climbs.

Sequence numbers and timestamps live in the fixed header rather than the extension block, but they are rewritten by the same code path and for the same reason: the subscriber must see one unbroken stream. Maintain a per-subscriber (seqOffset, tsOffset) pair and recompute it exactly once, at the switch boundary — never per packet. On a switch, seqOffset = lastEgressSeq + 1 - firstSeqOfNewSource, and tsOffset = lastEgressTs + frameDelta - firstTsOfNewSource, where frameDelta is one frame interval on the RTP clock (3000 ticks at 30 fps on video’s 90 kHz clock, 960 ticks per 20 ms Opus frame at 48 kHz).

The abs-send-time re-stamp and the timestamp offset are independent and must not be conflated. The RTP timestamp is a media presentation clock the subscriber uses to schedule playout; abs-send-time is a wall-clock departure stamp its estimator uses to measure one-way delay variation. Applying tsOffset to abs-send-time, or writing the media timestamp into the extension, produces a subscriber that renders correctly for a few seconds and then locks its estimate to the floor.

Reproduction Steps & Debugging Log Patterns

1. Negotiate deliberately mismatched extmap IDs. Publish from Chrome, then answer the subscriber with a hand-edited SDP that renumbers abs-send-time and mid — the mechanics of safely rewriting an answer are covered in the SDP Offer/Answer Lifecycle guide. A forwarder that caches one global ID table will now emit mid under the wrong ID and the subscriber’s transceiver goes silent while ICE stays connected.
2. Subscribe one client, then force a layer switch by throttling its downlink from 2 Mbps to 400 kbps for 5 s. Log the last egress sequence number before the switch and the first one after.
3. Open chrome://webrtc-internals on the subscriber and watch inbound-rtp. The tell for a bad abs-send-time is jitterBufferDelay climbing steadily while packetsLost stays at zero; the tell for a bad sequence offset is nackCount rising by hundreds per second with no matching loss.
4. Dump the first 32 bytes of ten consecutive egress packets and confirm the block still starts BE DE, that the word count matches the elements present, and that the transport-wide counter increments by exactly one across packets from different publishers on the same transport.

// Healthy switch: one offset recompute, no gap, transport counter untouched by the switch.
// [sub=7f2 mid=v0] fwd seq=20116 ts=7206000 twcc=41880 src=A1
// [sub=7f2 mid=v0] LAYER SWITCH A1->B2, seqOffset 15707 tsOffset 6696600
// [sub=7f2 mid=v0] fwd seq=20117 ts=7209000 twcc=41881 src=B2

// Broken: offset recomputed per packet, so the egress sequence walks backwards.
// [sub=7f2 mid=v0] fwd seq=20117 ts=7209000 twcc=41881 src=B2
// [sub=7f2 mid=v0] fwd seq=20114 ts=7212000 twcc=41882 src=B2   <-- non-monotonic
// [sub=7f2] NACK burst: 214 in 1s, packetsLost=0                <-- receiver chasing a phantom gap

// Broken: extension dropped instead of remapped (URI missing from egress extmap).
// [sub=7f2] ext uri=...abs-send-time ingressId=3 egressId=undefined -> dropped
// [sub=7f2] availableOutgoingBitrate stuck at 300000 for 30s

Poll getStats() on the subscriber at 1 s intervals and diff packetsReceived against your own egress counter; a persistent divergence of more than a handful of packets means the receiver is discarding packets your extension block made unparseable. If the layer that should be forwarded never appears at ingress in the first place, the fault is upstream of this code and Debugging Missing Simulcast Layers is the faster path.

Common Implementation Mistakes

• One global extmap table. Caching the publisher’s ID mapping and applying it to every egress works right up until a Firefox or Safari subscriber joins and numbers the same URIs differently. Store the mapping per transport, keyed by URI, and rebuild it on every renegotiation.
• Forwarding the publisher’s abs-send-time. The value then describes the publisher’s departure, not the server’s, so the first hop’s queueing shows up as constant jitter in the subscriber’s estimator and the estimate settles far below the real capacity. Always re-stamp immediately before encryption.
• Reusing the publisher’s transport-wide sequence number. The counter is per-transport and shared across every forwarded stream on it; copying it produces duplicates and reordering that make the feedback packets meaningless. Own a single 16-bit counter per egress transport.
• Leaving rid on egress. A subscriber that receives a rid for a layer it never subscribed to may bind the incoming SSRC to a phantom encoding, and Chrome will happily report the stream as active while rendering nothing. Strip it.
• Recomputing the sequence offset per packet. The offset must be derived once at the switch boundary from the last forwarded egress sequence; recomputing it on every packet makes the egress numbering follow the new source’s own numbering and walk backwards, triggering NACK storms. Align the boundary itself with the frame rules in Switching Layers Without Visible Glitches.
• Forgetting the block length after dropping elements. The extension header length counts 32-bit words; drop rid without re-padding and the receiver parses garbage where the payload should start.

FAQ

Do the ingress and egress extmap IDs have to match?

No, and expecting them to is the root cause of most extension bugs. Each RTCPeerConnection negotiates its own IDs, and browsers assign them in different orders. Translate through the canonical URI in both directions and treat any URI missing from the egress table as a drop, never as a pass-through with the ingress ID.

Does an SFU have to rewrite RTP timestamps as well as sequence numbers?

Yes, whenever the source behind a subscriber’s stream changes. Independent simulcast encodings start from unrelated random timestamp bases, so without a per-source offset the jitter buffer sees a jump of minutes and either stalls or flushes. Apply the offset on the media timestamp only, and pair the switch with the keyframe request described in Keyframe Request Strategies in an SFU.

What about the AV1 dependency descriptor — can I really copy it verbatim?

For layer selection you can, because the descriptor addresses frames within the bitstream and the forwarder is not renumbering those. You must still remap its extension ID, and if you drop temporal or spatial layers you have to drop whole frames on their dependency boundaries rather than individual packets; the layer structure that defines those boundaries is set up in Configuring AV1 SVC Layers in WebRTC.

Related: return to Selective Forwarding Unit Design, or read Bandwidth-Aware Layer Selection in an SFU for the logic that decides when these rewrites happen, and Reading chrome://webrtc-internals Dumps for reading the subscriber-side evidence.