Chunking Large File Transfers over Data Channels

Moving a 200 MiB video file peer-to-peer is not one send() call — it is a framing problem, a memory problem, and a progress-reporting problem stacked on top of a transport that has a hard per-message ceiling. This guide is part of the Data Channels & SCTP guide, and it settles one decision: what chunk size to use, how to frame and reassemble those chunks, and how to report honest progress while the transfer runs.

Context & Trade-offs

Every RTCDataChannel message has a maximum size, and it is negotiated rather than fixed. Chrome advertises a=max-message-size:262144 in the SDP — the ~256 KiB practical ceiling most engineers hit first. Firefox advertises a value in the gigabyte range. Safari and older endpoints advertise less or nothing at all, and when the attribute is absent the spec tells you to assume 64 KiB. Exceed whatever the remote peer actually advertised and the browser does not politely truncate: Chrome throws on send(), and some stacks abort the SCTP association outright, taking every other channel on that connection down with it.

That negotiated ceiling is a limit, not a target. Sending 256 KiB messages is legal against a modern Chrome peer and still wrong for three independent reasons. First, SCTP fragments a large message internally and only surfaces it to the application when the end-of-record fragment lands, so on an ordered channel a single 256 KiB message blocks delivery of everything queued behind it — including the control traffic you use to report progress. Second, the receiver cannot report meaningful progress: it learns about 256 KiB at a time, so a 20 MiB file produces only 80 progress ticks with long silences between them. Third, bufferedAmount becomes coarse; you cannot hold a 1 MiB high-water mark accurately when the quantum of work is a quarter of it.

16 KiB is the safe interoperable value, and the arithmetic behind it is boring in the best way. It is below every endpoint’s floor including the 16 KiB legacy cap, it is small enough that a 1 MiB send-buffer high-water mark holds 64 chunks in flight, and its framing overhead is negligible: a 12-byte header on a 16,384-byte message costs 0.07%. On a healthy path 16 KiB chunks sustain tens of megabytes per second — the limiting factor is SCTP’s congestion window and the shared path budget, never the chunk size.

Work the throughput arithmetic before you worry about the framing cost. A 50 MiB file at 16,372 payload bytes per chunk is 3,204 messages. At a modest 8 MB/s of goodput that is roughly 500 messages per second — half a millisecond of JavaScript per chunk to slice, prefix a header, and call send(), which is comfortably inside budget even on a mid-range phone. Push the chunk size down to 4 KiB and you quadruple the call rate to 2,000 messages per second, at which point the per-message SCTP bookkeeping and the event-loop churn start showing up in a profile. 16 KiB sits in the flat part of that curve: small enough for fine-grained progress and backpressure, large enough that per-message overhead disappears.

The second structural decision is to use two channels, not one. Chunks go on a dedicated reliable ordered channel; the manifest, acknowledgements, cancel requests, and completion notice go on a separate control channel. They share one SCTP association and one congestion window, but they are independent streams, so a control message never waits behind a megabyte of queued file bytes. Multiplexing both onto a single channel is the mistake that makes cancellation feel broken: the user hits stop, the cancel message lands behind 64 queued chunks, and the transfer keeps running for another second and a half.

The third trade-off is where the sequence numbers live. On a fully reliable ordered channel SCTP already guarantees order, so a receiver could simply append every arriving chunk. Do not rely on that alone. An explicit (transferId, seq) header costs 8 bytes, survives a concurrent second transfer on the same channel, lets you detect a truncated transfer instead of writing a silently short file, and makes the receiver’s progress counter authoritative rather than inferred. If you deliberately run the transfer over an unordered channel to avoid head-of-line blocking against other traffic — the same reasoning behind Reliable vs Unreliable Data Channels for Game State — the sequence number stops being belt-and-braces and becomes mandatory, because chunk 41 can legitimately arrive before chunk 39.

Minimal Runnable Implementation

The layout below uses a 12-byte binary header prefixed to each 16,372-byte payload slice, so the complete message is exactly 16,384 bytes. A small JSON manifest travels first and tells the receiver how much to expect.

const CHUNK = 16 * 1024;              // 16384 B total on the wire
const HEADER = 12;                    // transferId u32 | seq u32 | flags u8 | 3 B reserved
const PAYLOAD = CHUNK - HEADER;       // 16372 B of file bytes per message
const HIGH_WATER = 1024 * 1024;       // stop feeding SCTP above 1 MiB queued

const data = pc.createDataChannel('file', { ordered: true });   // reliable byte stream
const ctrl = pc.createDataChannel('file-ctrl', { ordered: true }); // manifest + acks
data.binaryType = 'arraybuffer';
data.bufferedAmountLowThreshold = 256 * 1024; // resume once the queue drains this far

// Join a leftover remainder with the next stream read, without copying twice.
const concat = (a, b) => { const o = new Uint8Array(a.byteLength + b.byteLength);
  o.set(a, 0); o.set(b, a.byteLength); return o; };

async function sendFile(file, transferId) {
  const total = Math.ceil(file.size / PAYLOAD);
  // Manifest first: the receiver cannot size its buffer or show a percentage without it.
  ctrl.send(JSON.stringify({
    type: 'manifest', transferId, name: file.name, size: file.size,
    mime: file.type, chunkSize: PAYLOAD, totalChunks: total
  }));

  const reader = file.stream().getReader();
  let seq = 0, carry = new Uint8Array(0), sentBytes = 0;

  const emit = async (bytes, final) => {
    const frame = new Uint8Array(HEADER + bytes.byteLength);
    const view = new DataView(frame.buffer);
    view.setUint32(0, transferId);          // groups chunks belonging to one transfer
    view.setUint32(4, seq++);               // strictly increasing; gaps mean loss or reorder
    view.setUint8(8, final ? 1 : 0);        // bit 0 marks the last chunk of the transfer
    frame.set(bytes, HEADER);               // payload follows the fixed-width header
    // Cooperative backpressure: never enqueue past the high-water mark.
    if (data.bufferedAmount > HIGH_WATER) {
      await new Promise((r) => {
        data.onbufferedamountlow = () => { data.onbufferedamountlow = null; r(); };
      });
    }
    data.send(frame);
    sentBytes += bytes.byteLength;
    // Honest progress: bytes handed to send() minus what SCTP has not drained yet.
    onProgress((sentBytes - data.bufferedAmount) / file.size);
  };

  for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    let buf = carry.byteLength ? concat(carry, value) : value; // keep frames exactly PAYLOAD
    let off = 0;
    while (buf.byteLength - off >= PAYLOAD) {
      await emit(buf.subarray(off, off + PAYLOAD), false);
      off += PAYLOAD;
    }
    carry = buf.subarray(off);              // remainder rides with the next stream read
  }
  await emit(carry, true);                  // final (possibly short) chunk carries the flag
}

The receiver mirrors this. It allocates one Uint8Array(size) from the manifest, copies each payload to seq * chunkSize, counts distinct sequence numbers rather than bytes so a duplicate cannot inflate progress, and only builds the Blob when the final-flag chunk has arrived and the received count equals totalChunks. Acknowledge every 64 chunks (1 MiB) on the control channel rather than per chunk — per-chunk acks add a message round trip for every 16 KiB and cost more than they tell you.

Two details in that loop are easy to get wrong. The carry buffer exists because ReadableStream reads return whatever size the platform feels like — typically 64 KiB from a File, but not reliably a multiple of your payload size — so without carrying the remainder forward you emit ragged frames and the receiver’s seq * chunkSize offsets drift. And the await sits before send(), not after: checking the queue once the byte is already enqueued lets a fast producer overshoot the high-water mark by an unbounded amount inside a tight synchronous loop.

Note the progress line: sentBytes - data.bufferedAmount. Counting only the bytes you passed to send() reports a transfer as 100% complete while up to a megabyte is still sitting in the SCTP queue, which is how progress bars end up frozen at “done” for several seconds. Subtracting bufferedAmount gives you bytes that have actually left the sender. The threshold plumbing behind that pause is covered in depth in Backpressure with bufferedAmountLowThreshold.

Reproduction Steps & Debugging Log Patterns

1. Create both channels before the first createOffer, or adding them later forces renegotiation through your SDP Offer/Answer Lifecycle handling mid-transfer.
2. Send a 50 MiB file (3,204 chunks at 16,372 B) and log seq, bufferedAmount, and the drained-progress percentage once per second.
3. Confirm the pause/resume cycle fires: bufferedAmount should oscillate between the 256 KiB low threshold and the 1 MiB high-water mark, never climbing monotonically.
4. Poll pc.getStats() at 1 s intervals and diff the data-channel report’s bytesSent against the receiver’s bytesReceived; on a reliable channel messagesSent must equal messagesReceived exactly.
5. Re-run with the sender throttled to a slow link and verify the receiver’s ack sequence still advances — a stalled ack stream with a growing bufferedAmount means the association is congestion-limited, not that your loop is broken.

// t+1s  seq=1024 buffered=917504 drained=31.4%   <- pausing at the high-water mark
// t+2s  seq=1024 buffered=262144 drained=32.7%   <- onbufferedamountlow fired
// t+3s  seq=2088 buffered=786432 drained=61.2%
// t+4s  ack received=2048 of 3204
// t+6s  seq=3204 buffered=0      drained=100%    <- only now is the transfer really done
// receiver: chunks=3204 unique=3204 bytes=52428800 final=true -> Blob assembled

Read that log for shape, not for absolute numbers. The signature of a healthy transfer is seq advancing in bursts while buffered saws between the two thresholds and drained climbs steadily; the signature of a missing await is seq sprinting to the final chunk in a few hundred milliseconds while buffered grows into the tens of megabytes and the tab’s memory graph goes vertical. A third pattern — seq frozen, buffered pinned just under the high-water mark, drained flat for several seconds — is neither bug: it is the path itself, and the fix is on the network rather than in the loop.

For a deeper read on the counters, Reading chrome://webrtc-internals Dumps shows where the data-channel and sctp-transport reports live, and Interpreting getStats() for Congestion Signals explains why a flat bytesSent line during a rising queue is congestion rather than an application bug.

Common Implementation Mistakes

• Sizing chunks from the advertised maximum. Reading a=max-message-size and sending 256 KiB messages works against one Chrome peer and breaks against Safari, a native endpoint, or an SFU-side gateway. Send 16 KiB and never read the attribute at all.
• Reporting progress from send() calls. Progress computed from bytes enqueued hits 100% up to a megabyte early. Always subtract bufferedAmount so the bar tracks bytes that have actually left the machine.
• Reassembling by append order without a sequence number. It happens to work on an ordered channel and fails silently the moment a second transfer shares the channel or you switch to ordered: false. Write each payload to seq * chunkSize instead.
• Buffering the whole file into one ArrayBuffer on both ends. A 1 GiB transfer needs 1 GiB of heap on the receiver and will kill a mobile tab. Stream to the File System Access API or IndexedDB once the manifest size exceeds a few hundred megabytes.
• Treating the final chunk as an implicit end. A transfer truncated by a closed channel looks exactly like a completed one if you only count bytes. Require both the final flag and unique === totalChunks before you hand a Blob to the application.
• Acking every chunk. Per-chunk acknowledgements add a control message per 16 KiB, competing with the file bytes for the same congestion window. Ack every 64 chunks or every 500 ms.

FAQ

Should I chunk on an ordered or unordered channel?

Ordered and fully reliable is the right default for files: SCTP delivers chunks in sequence, your receiver writes them contiguously, and the sequence number is only a consistency check. Choose ordered: false only when the same connection carries latency-sensitive traffic that must not queue behind file bytes, and accept that you then need the offset-based writes described above.

Can I use a bigger chunk if both peers are Chrome?

You can, and the throughput gain is small enough not to be worth it. The congestion window, not the message size, sets the ceiling on a healthy path, while larger messages cost you progress granularity and coarser backpressure. Keep 16 KiB and spend the effort on the send loop instead.

What happens to an in-flight transfer during an ICE restart?

The SCTP association rides the same DTLS transport, so a successful restart resumes the transfer with the queue intact — chunks pause and then continue. A full connection failure closes the channel, at which point the receiver has a contiguous prefix of the file and can request a resume from the highest contiguous seq it acked, which is exactly why the ack stream is worth keeping.

Related: return to the parent Data Channels & SCTP guide for the full reliability matrix, and see WebSocket Signaling Implementation for the offer/answer plumbing that must complete before the first chunk can move.