Audio Processing & Opus Tuning: AEC, Noise Suppression and Codec Parameters

Video gets the bandwidth budget and most of the engineering attention, but audio gets the complaints. Participants tolerate a frozen video tile for two seconds without saying a word; they abandon a call after ten seconds of robotic concealment artifacts or half a second of one-way delay. The frustrating part is that the audio path is almost entirely configurable — every stage between the microphone ADC and the RTP packet is a knob you either set deliberately or inherit as a browser default that was tuned for a speakerphone in an open-plan office. This guide is part of the Media Handling, Codecs & Bandwidth Estimation guide, and it covers the two halves of that path: the getUserMedia audio processing module (APM) that cleans the capture signal, and the Opus a=fmtp parameters that decide how much of the cleaned signal actually survives the network.

The implementation goal is concrete and testable: pick a capture profile whose processing matches the content, negotiate an Opus configuration that spends a defensible number of kilobits per second on it, push both through the sender, and then prove from getStats() that the encoder is producing the bitrate you asked for and the receiver’s jitter buffer is not silently adding 200 ms of delay to hide network jitter. Opus spans 6–510 kbps and every value in that range is legal, so “it sounds fine on my laptop” is not a configuration.

Step 1 — Choose a capture profile

getUserMedia({ audio: true }) is not a neutral request. It engages the full WebRTC audio processing module: a high-pass filter, acoustic echo cancellation, noise suppression, and automatic gain control, all running on 10 ms frames at 48 kHz before a single sample reaches the encoder. Each stage is individually addressable through a constraint, and each one trades a specific cost for a specific benefit.

Echo cancellation is the expensive one and the one you almost never turn off in a conversational context. Chrome’s AEC3 maintains an adaptive filter over the far-end reference signal and subtracts the estimated echo from the microphone input; it costs roughly 3–6% of a modern core per stream and contributes most of the 10–20 ms of algorithmic delay the APM adds before the encoder sees anything. Noise suppression estimates a stationary noise floor and applies spectral gain reduction — cheap at 1–2% of a core, but it is a speech-optimised model, so it treats sustained musical tones, applause, and room ambience as noise to be removed. Automatic gain control normalises level toward a target loudness with a slow attack; it costs well under 1% of a core but is the stage most likely to produce audible pumping, and on a high-gain condenser microphone it can drive the AEC filter toward divergence.

In practice you want two or three named profiles rather than one constraint object sprinkled with conditionals. A conversational profile leaves all three processors on. A presenter profile keeps echo cancellation but drops noise suppression and gain control so a properly gain-staged USB interface passes through untouched. A high-fidelity profile disables everything and requests stereo, which is the configuration explored in Disabling Audio Processing for High-Fidelity Music. Whichever you pick, read the settings back — constraints are requests, not commands, and the gap between what you asked for and what the platform granted is where most “my configuration does nothing” bugs live.

// Named capture profiles. Each is a plain MediaTrackConstraints object so it
// can be logged, diffed against getSettings(), and stored per user preference.
const AUDIO_PROFILES = {
  conversational: {
    echoCancellation: true,   // AEC3: mandatory whenever a loudspeaker is in the room
    noiseSuppression: true,   // strips keyboard clatter and HVAC hum
    autoGainControl: true,    // normalises laptop mics that sit 60 cm from the mouth
    channelCount: 1,          // the APM is mono-only; asking for 2 here wastes bits
    sampleRate: 48000
  },
  presenter: {
    echoCancellation: true,   // still needed: presenters monitor on speakers
    noiseSuppression: false,  // preserve breath and room tone from a good mic
    autoGainControl: false,   // the interface is already gain-staged to -18 dBFS
    channelCount: 1,
    sampleRate: 48000
  },
  highFidelity: {
    echoCancellation: false,  // requires a closed headset or you will echo
    noiseSuppression: false,
    autoGainControl: false,
    channelCount: 2,          // genuine stereo source, not a duplicated mono mic
    sampleRate: 48000
  }
};

async function captureAudio(profileName) {
  const wanted = AUDIO_PROFILES[profileName];
  const stream = await navigator.mediaDevices.getUserMedia({ audio: wanted });
  const track = stream.getAudioTracks()[0];

  // Constraints are requests. Compare granted settings against the request and
  // log every divergence — this is where iOS and Bluetooth surprises show up.
  const got = track.getSettings();
  for (const [key, value] of Object.entries(wanted)) {
    if (got[key] !== undefined && got[key] !== value) {
      console.warn(`audio profile ${profileName}: asked ${key}=${value}, got ${got[key]}`);
    }
  }
  return { stream, track, settings: got };
}

Device selection interacts with this more than people expect: the same constraint set produces a different granted configuration on a USB interface than on a laptop array microphone, and the deviceId handling that keeps those stable is covered in Media Constraints & Device Enumeration. Chrome honours all three boolean flags on desktop and Android. Firefox honours them but applies its own AGC curve. Safari on iOS is the outlier — the platform voice-processing audio unit engages whenever a microphone is captured, so echoCancellation: false is frequently granted in getSettings() while processing continues underneath.

Step 2 — Set the Opus fmtp parameters

Once the signal is clean, the codec decides what survives. Opus in WebRTC is always negotiated at 48 kHz with a dynamic payload type, and every meaningful tuning knob lives in the a=fmtp line rather than in any JavaScript API. The critical semantic — the one that makes half of all Opus tuning attempts no-ops — is that fmtp parameters are receiver-declared capabilities. When you put maxaveragebitrate=32000 in your own local description, you are telling the remote encoder how much to send you. It has no effect on your own outbound stream. Getting this backwards produces the classic symptom: an SDP full of correct-looking parameters and an outbound audio stream still pinned at the browser default.

The bitrate range deserves a moment of respect. Opus is legal from 6 kbps to 510 kbps, and the shape of the quality curve is steep at the bottom and almost flat at the top. Below roughly 12 kbps you are in narrowband territory where speech is intelligible but tiring. The 24–32 kbps band is the sweet spot for conversational voice: at 32 kbps mono, Opus at fullband is transparent enough that listeners cannot reliably pick it out of a lineup against the uncompressed source in a conversational context. Above 64 kbps you are paying for music, applause, and stereo image, not for speech clarity, and above about 160 kbps stereo the returns are inaudible outside a mastering room.

maxplaybackrate is the parameter most teams never touch and should. It caps the audio bandwidth the decoder will reconstruct, and Opus’s encoder uses it as a hint to stop spending bits above that frequency. Setting maxplaybackrate=16000 on a speech-only conference tells the far side to encode wideband rather than fullband, which frees bits for the frequency range that carries intelligibility instead of the range that carries hiss.

Packet size is the parameter with the least obvious cost. At the default ptime=20, the encoder emits 50 packets per second; each carries roughly 54 bytes of RTP, UDP, IPv4 and Ethernet headers plus a 10-byte SRTP authentication tag, which is about 25 kbps of pure overhead. Halve ptime to 10 ms to shave a frame of latency and that overhead doubles to about 50 kbps — more than the payload itself on a 24 kbps voice stream. Going the other way, ptime=60 cuts overhead to roughly 8 kbps but makes every lost packet cost 60 ms of audio, which in-band FEC then has to repair over a longer horizon. Twenty milliseconds is the default because it is the right answer nearly always.

; audio-profiles.ini — the signalling layer expands one section into the
; a=fmtp line it appends to the Opus payload type in the local description.
; Remember: these are RECEIVE capabilities. They shape what the peer sends you.

[voice]
maxaveragebitrate = 24000    ; 24 kbps mono, the conversational default
maxplaybackrate   = 16000    ; wideband; bits above 8 kHz do not aid intelligibility
useinbandfec      = 1        ; carry a coarse copy of frame N-1 inside frame N
usedtx            = 1        ; stop transmitting during silence in multi-party calls
stereo            = 0        ; mono decode; halves the bitrate for the same quality
cbr               = 0        ; keep VBR — CBR pays full rate for silence
ptime             = 20       ; 50 packets/s, ~25 kbps of header overhead

[content]
maxaveragebitrate = 160000   ; music and applause need the fullband headroom
maxplaybackrate   = 48000    ; full 20 kHz reconstruction
useinbandfec      = 0        ; FEC steals bits from the primary encoding at this rate
usedtx            = 0        ; DTX gates musical decay tails as if they were silence
stereo            = 1        ; declare stereo decode capability
sprop-stereo      = 1        ; and announce that we will also send stereo
ptime             = 20       ; do not shorten; header overhead is pure waste here

Setting usedtx=1 is a negotiation, not a local switch, and the mechanics of editing that single token safely are covered in Munging SDP to Prefer Opus DTX. The interaction between useinbandfec and the actual measured loss rate — including when FEC starts costing more quality than it recovers — is the subject of Tuning Opus Bitrate and FEC for Lossy Networks.

Step 3 — Wire the profile through the sender

Three separate mechanisms have to agree before a tuned profile reaches the wire, and they operate at different layers. setCodecPreferences() on the transceiver decides which codec wins the negotiation. The fmtp edit on the local description decides what the remote encoder aims for. setParameters() on the RTCRtpSender decides the hard ceiling on your own outbound stream. Skip any one of them and the other two quietly under-deliver.

Order matters. Call setCodecPreferences() before createOffer(), because it filters the codec list the offer is generated from. Apply the fmtp edit after createOffer() and before setLocalDescription(), so the committed description and the transmitted description are byte-identical. Apply setParameters() only after the connection reaches connected — Safari in particular ignores audio encoding changes made while the transport is still gathering.

// Build the fmtp token list from a profile object, keyed off the negotiated
// payload type. Rebuilding the whole token list (rather than appending one
// token) keeps the line deterministic across repeated renegotiations.
function applyOpusFmtp(sdp, profile) {
  const rtpmap = sdp.match(/a=rtpmap:(\d+) opus\/48000\/2/i);
  if (!rtpmap) return sdp;                       // no Opus offered — leave SDP alone
  const pt = rtpmap[1];                          // dynamic PT; never hardcode 111

  const line = new RegExp(`a=fmtp:${pt} ([^\\r\\n]*)`);
  const existing = sdp.match(line);
  const tokens = new Map();
  if (existing) {
    for (const pair of existing[1].split(';')) { // preserve params we do not manage
      const [k, v] = pair.split('=');
      if (k) tokens.set(k.trim(), v);
    }
  }
  for (const [k, v] of Object.entries(profile)) tokens.set(k, String(v));
  const rebuilt = [...tokens].map(([k, v]) => `${k}=${v}`).join(';');

  return existing
    ? sdp.replace(line, `a=fmtp:${pt} ${rebuilt}`)
    : sdp.replace(rtpmap[0], `${rtpmap[0]}\r\na=fmtp:${pt} ${rebuilt}`);
}

async function negotiateAudio(pc, track, profile) {
  const tx = pc.addTransceiver(track, { direction: 'sendrecv' });

  // 1. Put Opus first so nothing negotiates down to PCMU/G.722 on a bad day.
  const caps = RTCRtpSender.getCapabilities('audio');
  const ordered = [
    ...caps.codecs.filter(c => /opus/i.test(c.mimeType)),
    ...caps.codecs.filter(c => !/opus/i.test(c.mimeType))
  ];
  if (tx.setCodecPreferences) tx.setCodecPreferences(ordered);

  // 2. Declare our receive capabilities in the offer we are about to commit.
  const offer = await pc.createOffer();
  offer.sdp = applyOpusFmtp(offer.sdp, profile);
  await pc.setLocalDescription(offer);           // commit, then send localDescription

  // 3. Cap our OWN outbound stream. This is the only knob that binds locally.
  pc.addEventListener('connectionstatechange', async () => {
    if (pc.connectionState !== 'connected') return;
    const params = tx.sender.getParameters();
    if (!params.encodings?.length) params.encodings = [{}];
    params.encodings[0].maxBitrate = profile.maxaveragebitrate; // bps, matches fmtp
    params.encodings[0].networkPriority = 'high'; // audio before video under DSCP
    await tx.sender.setParameters(params);        // Chrome/Firefox honour this for audio
  });

  return tx;
}

Setting networkPriority: 'high' on the audio encoding is worth the one line. It marks the outgoing packets with the higher DSCP class, and more importantly it tells the sender’s own pacer to schedule audio ahead of video when the estimate collapses — which is exactly what you want, because a call with degraded video and clean audio is still a call. If the audio track will be swapped mid-session for a different device, keep the same sender rather than renegotiating; the routing and echo-recalibration side of that swap is detailed in Audio Focus & Echo Cancellation Across Devices.

Step 4 — Verification with getStats() audio metrics

Nothing above is real until the stats agree. Audio verification splits into three questions, each answered by a different report type: is the capture chain doing what I configured (media-source), is the encoder producing the bitrate I negotiated (outbound-rtp), and is the receiver’s jitter buffer paying for network instability with delay (inbound-rtp).

The media-source report for an audio track exposes audioLevel, totalAudioEnergy, and — when echo cancellation is active — echoReturnLoss and echoReturnLossEnhancement. ERLE is the direct measurement of how much echo the canceller is removing; a healthy converged AEC3 sits above roughly 20 dB and drops sharply for a second or two after a route change while the filter re-adapts. On outbound-rtp the field that matters is targetBitrate, which should land within a few percent of the maxaveragebitrate you negotiated once speech is flowing; if it sits far below, either DTX is active during silence or the sender-side ceiling from Step 3 is clamping it.

The receiver side is where perceived latency is actually decided, and the component responsible is NetEq — the adaptive jitter buffer that sits between arriving RTP packets and the audio output device. NetEq does not simply buffer; it continuously estimates the arrival-time distribution, computes a target delay, and then time-scales the audio to steer the actual buffer level toward that target. When the buffer has grown too deep it runs accelerate, discarding roughly 10 ms of audio by time-compressing a segment without changing pitch. When the buffer is running dry during a pause it runs preemptive expand, stretching a segment to buy time before an underflow. When a packet simply does not arrive it runs expand — packet loss concealment synthesised from the previous frame’s spectral model — and when the late packet finally shows up, merge splices it back in without a click.

This is why “audio latency” is not a single number you can read. The useful metric is the ratio jitterBufferDelay / jitterBufferEmittedCount, which gives the mean delay in seconds that each emitted sample spent waiting. Divide the counters between two polls rather than using the cumulative totals, or you will be averaging over the whole session and miss the moment the buffer doubled. Alongside it, track concealedSamples against totalSamplesReceived: under 1% is inaudible, above 2% is the warbling that users describe as “you’re breaking up.” The two move in opposition — NetEq will always trade one for the other — and separating a genuine capacity problem from a buffer that is merely being conservative is the analysis walked through in Measuring Audio Latency and Jitter Buffer Delay.

// Poll audio health at 1 s intervals and derive rates from counter deltas.
let prev = null;

async function sampleAudioHealth(pc) {
  const stats = await pc.getStats();
  const now = { t: performance.now() };

  for (const r of stats.values()) {
    if (r.type === 'media-source' && r.kind === 'audio') {
      now.level = r.audioLevel;                      // 0..1, instantaneous
      now.erle = r.echoReturnLossEnhancement;        // dB; >20 = AEC converged
    }
    if (r.type === 'outbound-rtp' && r.kind === 'audio') {
      now.bytesSent = r.bytesSent;
      now.targetBitrate = r.targetBitrate;           // compare to maxaveragebitrate
    }
    if (r.type === 'inbound-rtp' && r.kind === 'audio') {
      now.jbDelay = r.jitterBufferDelay;             // cumulative seconds
      now.jbEmitted = r.jitterBufferEmittedCount;    // cumulative samples
      now.jbTarget = r.jitterBufferTargetDelay;      // what NetEq is aiming for
      now.concealed = r.concealedSamples;
      now.totalSamples = r.totalSamplesReceived;
      now.fecRecovered = r.fecPacketsReceived;       // in-band FEC actually used
    }
  }

  if (prev) {
    // Mean buffer delay for samples emitted since the previous poll, in ms.
    const emitted = now.jbEmitted - prev.jbEmitted;
    const meanDelayMs = emitted > 0
      ? ((now.jbDelay - prev.jbDelay) / emitted) * 1000
      : 0;
    // Concealment ratio over the same window: >2% is audibly robotic.
    const samples = now.totalSamples - prev.totalSamples;
    const concealRatio = samples > 0
      ? (now.concealed - prev.concealed) / samples
      : 0;
    // Actual send rate, to confirm the negotiated fmtp is being honoured.
    const kbps = ((now.bytesSent - prev.bytesSent) * 8) /
      ((now.t - prev.t) * 1000);

    console.log(
      `audio: ${kbps.toFixed(1)} kbps, jb ${meanDelayMs.toFixed(0)} ms ` +
      `(target ${(now.jbTarget * 1000).toFixed(0)} ms), ` +
      `conceal ${(concealRatio * 100).toFixed(2)}%, ERLE ${now.erle ?? 'n/a'} dB`
    );
  }
  prev = now;
}

setInterval(() => sampleAudioHealth(pc), 1000); // 1 s cadence, same as video stats

A converged voice session with a [voice] profile should log something close to 24–28 kbps while speaking and a few hundred bps during silence if DTX negotiated, a jitter buffer delay of 40–80 ms on wired or stable Wi-Fi, a concealment ratio under 1%, and ERLE above 20 dB. Audio counters live in the same report set as the congestion signals, so the polling loop above slots straight into whatever you already run for Interpreting getStats() for Congestion Signals rather than needing its own timer.

Edge Cases & Browser Quirks

• Chrome clamps mono Opus regardless of your ceiling. Chrome’s own encoder settles around 32 kbps mono for speech even when the remote fmtp asks for more; raising it meaningfully requires stereo=1 in the remote description or a maxaveragebitrate well above the clamp, plus maxBitrate on the sender. Setting only one of the two produces no change, which is why this reads as “Chrome ignores maxaveragebitrate.”
• The legacy goog constraints are gone. googEchoCancellation, googNoiseSuppression, googHighpassFilter and friends were removed from Chrome in the M92 era. Code still passing them gets no error and no effect — the standard echoCancellation / noiseSuppression / autoGainControl booleans are the only supported surface.
• iOS Safari overrides your capture profile. Capturing a microphone on iOS engages the platform voice-processing unit. getSettings() may faithfully report echoCancellation: false while the OS continues to apply echo cancellation, gain control, and its own noise gate. Treat iOS high-fidelity capture as unavailable in Safari rather than debugging it.
• Bluetooth HFP collapses the whole chain to narrowband. The moment a Bluetooth headset’s microphone is opened, most platforms switch the link from A2DP to the HFP/SCO profile, which is 8 kHz or 16 kHz end to end. A maxplaybackrate=48000 profile is meaningless on that route; verify with track.getSettings().sampleRate and fall back to the voice profile.
• Firefox reports different AEC internals. Firefox implements the standard booleans but does not expose echoReturnLossEnhancement as consistently as Chrome, and its AGC target differs audibly. Build dashboards that tolerate undefined for ERLE instead of treating it as zero, and validate loudness against totalAudioEnergy instead.
• DTX breaks naive server-side recorders. A recorder that reconstructs a timeline by counting packets rather than reading RTP timestamps will compress every DTX silence gap out of the file, desynchronising audio from video by exactly the accumulated silence. Either disable usedtx on recorded sessions or fix the recorder to honour timestamps.
• ptime is advisory. Browsers treat it as a hint. Chrome honours 20 ms and will accept 10 ms via minptime, but neither Chrome nor Safari reliably follows a request for 40 or 60 ms, so do not build a bandwidth budget that depends on longer packets.

Common Implementation Mistakes

• Putting fmtp parameters in the wrong description. maxaveragebitrate in your local SDP governs what the peer sends you. Teams set it locally, watch their own outbound bitrate not move, and conclude the parameter is broken.
• Disabling all three processors to “reduce latency.” The APM contributes 10–20 ms. Turning off echo cancellation to reclaim it on a device with an open loudspeaker trades 15 ms of delay for a howling feedback loop.
• Hardcoding payload type 111. Opus is a dynamic payload type. Read it from a=rtpmap every negotiation, because it differs between engines and can change across a renegotiation.
• Requesting channelCount: 2 for a mono microphone. The browser duplicates the single channel across both, so you pay a stereo bitrate to transmit two identical signals — and the APM downmixes it anyway.
• Using CBR to make bandwidth planning easier. cbr=1 pays the full negotiated rate through silence, defeats DTX entirely, and gives Opus no room to spend bits where they matter. Plan capacity from measured VBR averages instead.
• Treating jitterBufferDelay as milliseconds. It is cumulative seconds across all emitted samples and is meaningless without dividing by jitterBufferEmittedCount. Dashboards that plot it raw show a line that only ever rises.
• Tuning by ear on a wired desktop. Every interesting audio failure — SCO fallback, focus arbitration, cellular jitter, AEC divergence — happens on a handset. Validate profiles on real mobile hardware before shipping them.

FAQ

What bitrate should I actually set for a voice call?

Start at 24 kbps mono with useinbandfec=1 and maxplaybackrate=16000, and raise to 32 kbps if listeners report thinness. That band is the industry default because Opus’s quality curve flattens above it for speech: going from 24 to 48 kbps roughly doubles cost for a difference most listeners cannot identify in conversation, while going from 12 to 24 kbps is dramatic.

Does noise suppression hurt audio quality?

For speech, no — it removes stationary noise the listener did not want. For anything non-speech it is destructive, because the model treats sustained tones, reverb tails, and applause as noise and gates them. If a user complains that their guitar sounds like it is underwater, noise suppression is almost always the cause, not the codec.

Why is my measured audio latency 200 ms when the network RTT is 40 ms?

Because NetEq is buying stability with delay. On a jittery path it raises jitterBufferTargetDelay so late packets still arrive before their playout deadline, and that target — not the RTT — dominates end-to-end audio delay. Check jitterBufferTargetDelay against jitterBufferDelay / jitterBufferEmittedCount; if both are high while the concealment ratio is near zero, the buffer is deliberately trading latency for smoothness.

Should I enable DTX in a two-party call?

Rarely worth it. DTX saves 24–32 kbps per silent stream, which matters when a room has twenty participants and nineteen are muted, but in a two-party call it saves a rounding error while adding a small risk of a clipped word onset at the speech boundary. Enable it for multi-party sessions and leave it off for one-to-one.

Related: this section sits under the Media Handling, Codecs & Bandwidth Estimation guide; for the loss-resilience side see Tuning Opus Bitrate and FEC for Lossy Networks, for non-speech content see Disabling Audio Processing for High-Fidelity Music, and for the delay budget itself see Measuring Audio Latency and Jitter Buffer Delay.