Dynamically Switching Video Codecs Based on Client Capabilities

Programmatically changing video codecs mid-session requires precise SDP manipulation and RTCRtpSender parameter updates. This approach prevents negotiation failures and optimizes bandwidth allocation.

Capability Detection & SDP Payload Filtering

Enumerate supported codecs using RTCRtpSender.getCapabilities('video') before peer connection setup. Filter out unsupported payloads to guarantee successful negotiation.

Evaluate hardware-accelerated decoders against software fallbacks using VP8 vs H264 vs AV1 Codec Selection for cross-browser compatibility. Modify the generated SDP to reorder or remove codec lines before calling setLocalDescription.

Reproduction Steps:

1. Call RTCRtpSender.getCapabilities('video') on a Chromium-based browser.
2. Log the returned codecs array and identify valid mimeType entries.
3. Apply a filter to remove video/AV1 if hardware acceleration is unavailable.
4. Generate an offer with createOffer() and verify the SDP m=video line reflects the filtered order.

Watch for these logs:

• RTCErrorEvent: Failed to parse SDP: unsupported codec payload
• getCapabilities() returned empty mimeType array

const capabilities = RTCRtpSender.getCapabilities('video');
const preferredOrder = ['video/VP8', 'video/H264', 'video/AV1'];
const filteredCodecs = capabilities.codecs.filter(c => preferredOrder.includes(c.mimeType));

const offer = await pc.createOffer();
const sdpLines = offer.sdp.split('\n');
const mLineIndex = sdpLines.findIndex(l => l.startsWith('m=video'));
const newMLine = `m=video 9 UDP/TLS/RTP/SAVPF ${filteredCodecs.map(c => c.payloadType).join(' ')}`;
sdpLines[mLineIndex] = newMLine;
await pc.setLocalDescription({ type: 'offer', sdp: sdpLines.join('\n') });

Mid-Call Codec Switching via setParameters

Dynamic switching during an active session requires intercepting RTCRtpSender.setParameters(). Extract the current encodings array and modify the codecPayloadType to match your target codec.

This method bypasses full renegotiation but demands strict state management. Reference broader Media Handling, Codecs & Bandwidth Estimation workflows to maintain connection stability. Always trigger a keyframe request immediately after switching to prevent decoder desync.

Reproduction Steps:

1. Establish an active WebRTC session using VP8.
2. Call sender.getParameters() on the active video track.
3. Replace codecs[0].payloadType with the target codec’s payload ID.
4. Execute sender.setParameters(updatedParams) and monitor for InvalidStateError.
5. Call sender.generateKeyFrame() and verify frame continuity via getStats().

Watch for these logs:

• setParameters failed: Invalid codec payload type
• RTCPeerConnection: ICE connection state changed to checking

async function switchCodec(sender, targetMimeType) {
 const params = sender.getParameters();
 const targetCodec = RTCRtpSender.getCapabilities('video').codecs.find(c => c.mimeType === targetMimeType);
 if (!targetCodec) throw new Error('Codec not supported');
 
 params.codecs = [targetCodec, ...params.codecs.filter(c => c.mimeType !== targetMimeType)];
 await sender.setParameters(params);
 await sender.generateKeyFrame();
}

Fallback Handling & Congestion Triggers

Monitor network degradation by polling getStats() for frameRate and packetsLost. Trigger a downgrade to a more resilient codec when packet loss exceeds 5%.

Revert to baseline VP8 if degradation persists. Ensure the encoder state resets cleanly to prevent artifacting during the transition. Log all transitions for telemetry and performance analysis.

Reproduction Steps:

1. Simulate 20% packet loss using tc or Chrome DevTools network throttling.
2. Poll getStats() every 2 seconds for packetsLost and framesDropped.
3. Trigger setParameters with a lower-complexity codec when thresholds are breached.
4. Verify a PLI/FIR is sent and the remote peer resumes decoding without freezing.

Watch for these logs:

• WebRTC: Encoder reconfiguration timeout
• stats: packetsLost > threshold, triggering codec fallback

Common Implementation Mistakes

• Modifying the SDP string after calling setLocalDescription, which violates the WebRTC spec and causes ICE/DTLS failures.
• Ignoring RTCRtpSender.getCapabilities() and hardcoding payload types, leading to cross-browser negotiation errors.
• Failing to send a keyframe request (generateKeyFrame()) after switching, resulting in prolonged green screens or decoder stalls.
• Triggering codec switches too frequently during transient network jitter, causing encoder re-initialization overhead and CPU spikes.

Frequently Asked Questions

Can I switch codecs without renegotiating the SDP? Yes. Modern browsers support RTCRtpSender.setParameters() to change the active codec payload type mid-call without triggering a full SDP renegotiation cycle. You must ensure the new codec is already present in the negotiated SDP m=video line.

Why does my video freeze immediately after calling setParameters? The remote decoder lacks a reference frame for the new codec stream. Always call RTCRtpSender.generateKeyFrame() immediately after setParameters() to force a clean IDR frame transmission and reset the decoder state.

How do I detect if a client supports AV1 hardware decoding? Use RTCRtpSender.getCapabilities('video') and inspect the codecs array. While the API does not explicitly expose hardware acceleration flags, you can infer support by checking for known AV1 payload types and cross-referencing with browser version matrices. Fallback to VP8 if performance metrics degrade.