Screen Sharing & Content Hints: Capturing and Encoding Screen Content

Screen capture is the only WebRTC source where the pixels are synthetic: perfectly sharp glyph edges, vast regions that do not change for minutes, and then a single keystroke that repaints the entire frame. This guide is part of the Media Handling, Codecs & Bandwidth Estimation guide, and it covers the full path from getDisplayMedia() and its constraint surface, through MediaStreamTrack.contentHint and the encoding parameters that make shared text legible, to the stats that prove it worked. The implementation goal is a share that holds a 1:1 pixel mapping of 12 px text at 1080p on roughly 800 kbps of steady-state uplink, absorbs a full-frame slide change in under 400 ms, and cleans itself up the instant the user hits the browser’s own Stop sharing button.

Step 1 — Request the capture and read back what you actually got

navigator.mediaDevices.getDisplayMedia() looks like getUserMedia() and behaves nothing like it. It requires transient user activation — a click or keypress within the last few seconds — so you cannot call it on page load, from a timer, or after an await that lets the activation expire. It never enumerates: there is no device ID to persist, no way to re-open the same surface silently on the next call, and no permission grant that survives the page. Every share is a fresh trip through the browser’s own picker, and the picker, not your constraint object, decides what you get.

That distinction is the source of most screen-sharing bugs. The constraint surface — displaySurface, logicalSurface, cursor, selfBrowserSurface, surfaceSwitching, systemAudio, monitorTypeSurfaces — is advisory in every shipping browser. Passing displaySurface: 'browser' does not restrict the user to a tab; it asks the picker to open on its tab pane and pre-select it. Wrapping it in { exact: 'browser' } is worse than useless: Chrome ignores the exact qualifier on display-surface constraints rather than throwing OverconstrainedError, so your code reads as enforcing something it is not. The only authoritative answer is videoTrack.getSettings() after the promise resolves.

Three of those options are worth setting on every request even though they are hints. selfBrowserSurface: 'exclude' removes your own tab from the picker, which prevents the infinite-hall-of-mirrors capture when a user shares the conferencing tab itself — a recursion that pushes encoder CPU past 100% within a second. surfaceSwitching: 'include' gives Chrome’s share bar a Change source control, letting the user swap from a slide deck to a terminal without a second getDisplayMedia() round trip; the track object survives, so no renegotiation happens, but its resolution changes underneath you. logicalSurface: true asks for the window’s logical backing surface rather than the on-screen composited pixels, which is what keeps a partially occluded window capturing correctly instead of streaming whatever is stacked on top of it.

// Must be called synchronously enough after a click that transient activation is still valid.
async function startShare() {
  let stream;
  try {
    stream = await navigator.mediaDevices.getDisplayMedia({
      video: {
        displaySurface: 'browser',   // hint only: pre-selects the tab pane in the picker
        logicalSurface: true,        // capture the backing surface, not the occluded pixels
        cursor: 'motion',            // draw the pointer only while it moves
        frameRate: { ideal: 15, max: 30 } // ceiling, not a promise; see Step 2
      },
      audio: false,                  // system audio has its own rules — keep it explicit
      selfBrowserSurface: 'exclude', // never let the user pick this tab (feedback loop)
      surfaceSwitching: 'include'    // allow in-share source switching without a new prompt
    });
  } catch (err) {
    // NotAllowedError covers both "user cancelled the picker" and "policy blocked it".
    if (err.name === 'NotAllowedError') return null; // treat cancel as a no-op, not an error toast
    throw err; // NotFoundError / NotSupportedError are real bugs worth surfacing
  }
  const [videoTrack] = stream.getVideoTracks();
  const s = videoTrack.getSettings();
  // Branch on the real surface: a monitor share needs different framerate policy to a tab share.
  console.log(`surface=${s.displaySurface} ${s.width}x${s.height}@${s.frameRate}`);
  return { stream, videoTrack, surface: s.displaySurface };
}

Note the frame-rate constraint is a ceiling on the capturer, not a floor. Chrome’s screen capturer is change-driven: it delivers a frame when the surface repaints, so a motionless desktop with frameRate: { ideal: 15 } produces well under 1 fps and consumes almost no bitrate. That is the behaviour you want, and it is why raising the ceiling costs nothing on static content while a min constraint would force synthetic repeats of identical frames. Which surface the user picks changes the calculus enough that it is worth reading the trade-offs in Sharing a Tab vs a Window vs the Whole Screen before you decide what to pre-select.

Step 2 — Set contentHint and reshape the encoding parameters

A WebRTC video encoder shipped in a browser is tuned by default for a camera: a noisy, softly-focused, temporally-dithered 30 fps signal where every macroblock changes slightly every frame and the human eye forgives spatial detail loss in exchange for smooth motion. Screen content violates every one of those assumptions. Text is a high-frequency signal that lands exactly on the coefficients the quantiser discards first, so a rate controller that is happy to blur a face into a 28-quantiser mush turns 12 px glyphs into unreadable grey porridge. Chroma subsampling compounds it: 4:2:0 halves the colour resolution in both axes, and coloured syntax highlighting on a dark editor background is precisely the case where that shows as fringing. And the deblocking filter, designed to hide block boundaries in a camera picture, softens the crisp one-pixel edges that make text legible.

The temporal structure is just as hostile. For minutes at a time the frame is byte-identical and the encoder emits near-empty skip frames; then a slide advance or a full-page scroll invalidates 100% of the picture at once, requiring a burst of intra-coded blocks that can be 30–50× the size of a steady-state frame. A camera-tuned rate controller meets that burst by dropping resolution — exactly the wrong reaction, because it lands on the one frame the viewer is actually reading.

MediaStreamTrack.contentHint is the standard way to tell the encoder which trade-off to make. Four values apply to video tracks: "" (unset, browser guesses from source), "motion", "detail", and "text". The hint has no effect on the wire format and requires no renegotiation — it is a local instruction to the rate controller and the internal degradation policy.

Setting the hint alone is not enough, because contentHint biases the encoder’s internal choices but does not set the hard limits. Pair it with RTCRtpSender.setParameters(): set degradationPreference explicitly rather than trusting the hint to imply it, cap maxFramerate so the rate controller spends its budget on quality instead of redundant repaints, and pin scaleResolutionDownBy to 1 so a 1920×1080 surface arrives as 1920×1080. A half-scale share of a code editor is not a slightly worse share — it is an unusable one, because the receiver upscales 960×540 back to a 1080p element and every glyph stem lands between pixels. The single most valuable line is degradationPreference: 'maintain-resolution', and the broader reasoning behind that axis is set out in degradationPreference: Resolution vs Framerate.

// Apply after replaceTrack/addTrack, once the sender exists. No renegotiation is triggered.
async function configureScreenSender(sender, screenTrack, surface) {
  // Hint first: some encoders read it when the encoding session is (re)initialised.
  screenTrack.contentHint = surface === 'monitor' ? 'detail' : 'text'; // text for tab/window shares

  const params = sender.getParameters();
  if (!params.encodings?.length) params.encodings = [{}]; // Firefox can return an empty array

  // Never sacrifice pixels for frames on shared text.
  params.degradationPreference = 'maintain-resolution';

  const e = params.encodings[0];
  e.scaleResolutionDownBy = 1;   // 1:1 pixel mapping; anything > 1 destroys small glyphs
  e.maxFramerate = 8;            // static documents: 8 fps looks fine, scrolling stays readable
  e.maxBitrate = 1_800_000;      // headroom for the intra burst, not a steady-state target
  e.priority = 'high';           // bias the send-side allocator against a co-sent camera track

  await sender.setParameters(params); // slow control signal — do not call this per frame
}

Two numbers in that block deserve justification. maxFramerate = 8 sounds brutal until you remember the capturer is change-driven: 8 fps is a ceiling on repaints, and a static slide still costs near zero. What it buys is that when a scroll does happen, each of those eight frames gets roughly four times the bit budget it would have had at 30 fps, which is the difference between readable and smeared. Raise it to 30 and set contentHint = 'motion' only when the shared surface is genuinely playing video. And maxBitrate = 1.8 Mbps is deliberately far above the ~300–800 kbps a document share actually averages, because the ceiling exists to let the intra burst through without the pacer spreading one keyframe across 600 ms of queue. Setting a tight 600 kbps cap on a text share is the classic way to produce a share that is permanently one second behind the presenter.

Audio tracks carry their own hint vocabulary — "speech", "speech-recognition", and "music" — which matters if you are also pulling tab or system audio alongside the video, a path with enough platform-specific rules to justify its own treatment in Capturing System Audio with getDisplayMedia. If you want the per-glyph detail on quantiser behaviour, font rendering, and how to measure legibility rather than guess at it, that is the subject of Keeping Shared Text Readable with contentHint.

Step 3 — Handle the user stopping the share

Screen capture is the only media source with a second, browser-owned stop button. The floating Stop sharing bar, the picker’s own controls, closing the captured window, and locking the workstation all terminate the capture without your UI ever being clicked. When that happens the video track transitions to readyState === 'ended' and fires an ended event exactly once. If you only wire your own button, the sender keeps a dead track, remote peers see a frozen last frame indefinitely, and your UI still says “You are sharing.”

The self-transition is the subtle one. With surfaceSwitching: 'include', a user who switches from a 1280×800 window to a 3840×2160 monitor keeps the same track object and the same MediaStreamTrack.id. No ended, no mute, no renegotiation — but getSettings().width quadruples, and any scaleResolutionDownBy you computed from the old dimensions is now wrong. Chrome fires no standard event for this; the practical detection is a resize event on a <video> element wired to the stream, or a low-frequency settings poll on the same 1 s cadence you already use for stats.

function wireShareLifecycle(sender, screenTrack, cameraTrack, onUiChange) {
  let stopped = false;
  // One handler for every termination path: browser stop bar, window close, our own button.
  const finish = async () => {
    if (stopped) return; // 'ended' plus an explicit stop() must not run this twice
    stopped = true;
    screenTrack.stop();                       // idempotent; releases the OS capture handle
    await sender.replaceTrack(cameraTrack);   // same SSRC, no offer/answer round trip
    cameraTrack.contentHint = 'motion';       // undo the screen-share hint on the camera
    const p = sender.getParameters();
    p.degradationPreference = 'balanced';     // and undo maintain-resolution
    if (p.encodings?.[0]) p.encodings[0].maxFramerate = 30;
    await sender.setParameters(p);
    onUiChange({ sharing: false });           // only now flip the UI state
  };

  screenTrack.addEventListener('ended', finish); // fires exactly once, never re-armed
  // Surface switching keeps the same track: detect the new geometry by polling settings.
  let lastW = screenTrack.getSettings().width;
  const geometryTimer = setInterval(() => {
    if (screenTrack.readyState !== 'live') return clearInterval(geometryTimer);
    const w = screenTrack.getSettings().width;
    if (w !== lastW) { lastW = w; console.log('surface switched, re-evaluate scaling:', w); }
  }, 1000); // 1 s cadence matches the stats poll below

  return finish; // call this from your own Stop button too
}

Because replaceTrack() swaps the source on an existing sender, none of this touches SDP; the mechanics of that swap, including the cases where it legitimately rejects, are covered in Replacing Video Tracks Without Renegotiation.

Step 4 — Verification with getStats()

The failure mode you are hunting is silent: the share works, nobody complains loudly, and the text is 20% too soft to read comfortably. Poll getStats() at 1 s intervals and compare three pairs of numbers. First, media-source width and height against outbound-rtp frameWidth/frameHeight — if they differ, something downscaled the frame despite scaleResolutionDownBy = 1, and the culprit is almost always qualityLimitationReason. Second, qualityLimitationReason itself: bandwidth means the estimate is genuinely below what the surface needs, while cpu means the encoder cannot keep up at full resolution and the adaptation is local. Third, keyFramesEncoded deltas against bytesSent deltas, which is how you see the intra burst pass through.

// Poll once per second; compare the source geometry with what actually left the encoder.
async function auditShare(pc) {
  const report = await pc.getStats();
  let src = null, out = null;
  for (const r of report.values()) {
    if (r.type === 'media-source' && r.kind === 'video') src = r;      // pre-encode frames
    if (r.type === 'outbound-rtp' && r.kind === 'video') out = r;      // post-encode reality
  }
  if (!src || !out) return;

  const downscaled = out.frameWidth < src.width; // should be false with scaleResolutionDownBy = 1
  console.log({
    source: `${src.width}x${src.height}@${Math.round(src.framesPerSecond ?? 0)}`,
    sent: `${out.frameWidth}x${out.frameHeight}@${Math.round(out.framesPerSecond ?? 0)}`,
    downscaled,                                  // true => the hint or the params are being overridden
    reason: out.qualityLimitationReason,         // "none" | "bandwidth" | "cpu" | "other"
    keyframes: out.keyFramesEncoded,             // increments on every full-surface repaint
    // Rising encode time with a static surface means the encoder is re-coding identical frames.
    msPerFrame: out.framesEncoded ? (out.totalEncodeTime / out.framesEncoded * 1000).toFixed(1) : null
  });
}

Interpretation rules that hold across engines: qualityLimitationReason === 'cpu' on a static share almost always means the capturer is running at 30 fps on a 4K monitor and the encoder is burning cycles on frames that carry no new information — cap maxFramerate before you touch resolution. A frameWidth that is exactly half the source width means maintain-resolution did not take effect, which on Firefox usually traces to an empty encodings array. And keyFramesEncoded climbing steadily on an idle surface means something is requesting PLIs in a loop, typically a receiver that keeps failing to decode; the same congestion-versus-encoder disambiguation applies here as in Interpreting getStats() for Congestion Signals.

Edge Cases & Browser Quirks

• Safari has no display-surface filtering and no captured audio. Safari 17 and 18 on macOS support getDisplayMedia() but ignore displaySurface, selfBrowserSurface, and surfaceSwitching, and reject or silently drop the audio member — there is no tab audio and no system audio path. Safari on iOS does not implement getDisplayMedia() at all, so feature-detect rather than user-agent sniff, and hide the share button entirely on iOS.
• Firefox exposes no systemAudio or self-surface controls. Firefox honours displaySurface and cursor as picker hints and supports contentHint, but has no in-share surface switching and returns an empty encodings array from getParameters() on some senders — always guard before indexing, or setParameters() throws and your maintain-resolution never lands.
• Chrome’s system audio is platform-conditional. Chrome offers a “Share tab audio” checkbox for tab captures on all desktop platforms, but whole-screen system audio is Windows and ChromeOS only; the same code on macOS resolves with a video-only stream and no error. Check stream.getAudioTracks().length rather than assuming the request succeeded.
• cursor: 'always' costs real bitrate. A rendered pointer is a small, high-contrast, constantly moving region that defeats static-frame skipping — it can add 40–80 kbps of otherwise-idle traffic and, worse, keeps the encoder awake at your maxFramerate ceiling. Use 'motion' for presentations and 'never' when you are sharing a surface the viewer does not need to follow interactively.
• Occluded and minimised windows. Window capture on Windows can freeze on the last painted frame when the window is minimised, and on Wayland the portal may return a black surface if the compositor denies the restore token. logicalSurface: true helps on Chrome but is not a guarantee; detect via framesPerSecond dropping to 0 while readyState stays 'live'.
• HDR and 4K monitors. A 3840×2160 capture with scaleResolutionDownBy = 1 will exceed most software encoder budgets. For monitor shares above 2560 px wide, set scaleResolutionDownBy = 1.5 deliberately and accept the trade, rather than letting qualityLimitationReason: 'cpu' pick an arbitrary factor mid-sentence.
• Codec choice changes the text verdict. VP9 and AV1 handle sharp edges materially better than VP8 at the same bitrate, and AV1’s screen-content tools help most exactly on synthetic surfaces; whether you can rely on them depends on the receiving fleet, which is the trade explored in VP8 vs H.264 vs AV1 Codec Selection.

Common Implementation Mistakes

• Trusting the constraint instead of the settings. Requesting displaySurface: 'browser' and then writing code that assumes a tab was shared breaks the moment a user picks a monitor. Fix: branch on getSettings().displaySurface every time.
• Calling getDisplayMedia() after an await. Awaiting a network round trip inside the click handler consumes transient activation, and the call rejects with InvalidStateError or NotAllowedError depending on the engine. Fix: call it first, then do the async work with the resulting stream.
• Leaving degradationPreference at the default. The content hint alone does not reliably pin resolution under load in every Chromium version. Fix: set maintain-resolution explicitly on the sender.
• Sending screen content through camera simulcast encodings. Three spatial layers at 1×, 1/2, and 1/4 scale mean two of the three are illegible for text, and the SFU may well pick one of them. Fix: send a single high-resolution encoding for the share track, or restrict the layer choice on the server side.
• Only handling your own Stop button. The browser’s stop bar fires ended and nothing else; a UI that never listens leaves peers staring at a frozen frame. Fix: converge every path on one idempotent handler.
• Capping maxBitrate at the average. Sizing the ceiling to the ~500 kbps steady state starves the intra burst, so a slide change takes 1.5 s to resolve instead of 400 ms. Fix: size the ceiling for the burst and let the change-driven capturer keep the average low.
• Forgetting to reset encoder state after the share. A camera track inheriting maintain-resolution and maxFramerate: 8 from the previous screen share looks like a broken webcam. Fix: restore both when you swap back.

FAQ

Does setting contentHint require renegotiation? No. It is a local property on the track that influences encoder configuration and degradation policy only; the SDP, the codec, and the SSRC are unaffected. You can flip it mid-call and the change takes effect at the next encoder reconfiguration, typically within one or two frames.

What is the difference between "detail" and "text"? Both select spatial fidelity over frame rate and both map to resolution-preserving behaviour. "text" is the stronger signal, intended for surfaces that are predominantly rendered glyphs, and in Chromium it biases the rate controller further toward preserving high-frequency edges at a lower frame rate. In current builds the two behave similarly enough that either is acceptable for a document share, but use "text" for terminals, code editors, and spreadsheets so the intent survives future encoder changes.

Why does my share look sharp locally but soft to remote peers? The local preview renders the raw capture and never passes through the encoder. Compare media-source dimensions with outbound-rtp.frameWidth in getStats() — if they differ, the encoder downscaled and your local preview is simply not telling you. This is also why “it looks fine on my machine” is never evidence in a screen-share bug report.

Can I re-open the previous share without prompting the user again? No. There is no persistent permission and no device ID for a display surface, by design: every capture requires a fresh gesture and a fresh trip through the picker. The closest approximation is keeping the track alive across your own UI’s pause state instead of stopping it — but a paused-but-live capture keeps the browser’s sharing indicator visible, which users notice.

Related: this section sits under the Media Handling, Codecs & Bandwidth Estimation guide; go deeper with Capturing System Audio with getDisplayMedia, Keeping Shared Text Readable with contentHint, and Sharing a Tab vs a Window vs the Whole Screen, then pair them with Audio/Video Track Management and Adaptive Bitrate Streaming in WebRTC.