Measuring browser video freezes with frame callbacks

A streamer sees a face swap preview move for several seconds, freeze during a head turn and then catch up. A ten second summary still says 30 FPS. That number can be technically correct while leaving out the moment the streamer needs to diagnose.

The same question comes up when building a browser video tool. A connection can open successfully and produce a first visible frame, yet still fail to keep moving. To separate those cases, I would log startup time, frame count and the longest gap as separate observations. A short recording of the input and output then gives the viewer something concrete to compare.

Two traces with the same average

Take a fixed window of 10,000 milliseconds. In trace A, count 300 frames spaced at one thirtieth of a second. In trace B, count 150 frames in the first 4,500 milliseconds, then another 150 frames after a one second pause, over the final 4,500 milliseconds. Both traces contain 300 frames in ten seconds. The first has gaps of roughly 33.3 milliseconds. The second jumps from 4,500 to 5,530 milliseconds between consecutive frames, a gap of 1,030 milliseconds. These are synthetic timestamps, so the comparison is independent of a particular model, device or connection.

For a quick reproduction in a JavaScript console, use a fixed observation window and calculate the differences between consecutive timestamps. The timer below is a list of numbers. Running it uses zero webcam data and does not start a paid session.

const even = Array.from({ length: 300 }, (_, i) => (i + 1) * 1000 / 30);
const burst = Array.from({ length: 300 }, (_, i) =>
  (i + 1) * 30 + (i >= 150 ? 1000 : 0)
);

for (const [name, timestamps] of Object.entries({ even, burst })) {
  const gaps = timestamps.slice(1).map((t, i) => t - timestamps[i]);
  console.log(name, {
    fps: timestamps.length / 10,
    longestGapMs: Math.round(Math.max(...gaps)),
    gapsOver250Ms: gaps.filter(gap => gap > 250).length
  });
}

The cutoff here is an example to show why counting gaps adds information. Pick a threshold that suits the motion and context of your own application. A short pause between spoken sentences and a frozen transition during a live hand gesture may carry very different costs for a viewer.

Put the clock where the frame is displayed

In the browser, requestVideoFrameCallback lets you observe when a video frame is sent to the compositor. Record the callback time, the frame's media timestamp and, where useful, the change in its presented frame count. For each visible run, you can then report the time to the first observed frame and the spacing between subsequent observations. When the count jumps, flag that interval so you know the callback history may have missed a frame.

Keep the measurement tab in the foreground. If the tab becomes hidden, the video element is replaced or the user pauses playback, end the current observation window. Start a fresh one when the viewer is looking at the active preview again. The Page Visibility API provides a signal for the first case. This prevents time spent looking elsewhere from showing up as a freeze in the active output.

Frame callbacks run on the main thread, so a busy page or the browser's scheduling can delay the observation too. Treat a large gap as evidence that the preview needs a closer look. Correlate it with a local recording and, if your application exposes it, inbound video statistics. You can also compare the wall clock gap with the change in media timestamps to spot a case worth investigating. A callback gap alone cannot tell you how the camera, network, decoding and browser presentation each contributed.

Record three views in an order that answers one question

First watch the original camera preview, then the transformed output, then the recording from your capture or broadcast app. Keep the reference image, lighting and movement fixed between runs. Make one slow head turn and hold still briefly. The turn gives you visible motion to compare, and the hold gives you a point where it should settle. If you need a clear marker, bring a hand into the frame and lower it at the same point on each run.

If the original and output both hesitate at the same instant, simplify the camera input and browser workload before comparing models. Check whether another application is also using the camera and repeat with a simpler scene. If the input stays smooth and the output pauses, compare another pass on a stable connection while keeping the same movement. Jot down whether the output falls behind steadily, stops and jumps or remains fresh while the face changes shape. Those are three distinct observations with different follow up tests.

If the browser output stays smooth and the OBS recording stalls, look at how OBS captures and renders the scene. Try a local recording with the same capture source before starting a public stream. If the local recording looks smooth and the stream stutters for viewers, check OBS's network dropped frames separately. The OBS troubleshooting guide explains what those drops mean. Keep the local clip alongside the exported streaming statistics so the two observations can be checked together.

Keep a result someone else can compare

A useful report needs only a few details: which of the three views failed first, the observation window, the longest visible gap and what changed when you repeated one test. If the camera and output use a shared screen recording, compare their movement in that clip to estimate visible delay. Frame spacing alone describes continuity and cannot give you the time from camera movement to display.

Repeat the preflight in the browser and recording setup you actually plan to use. Save one short clip where you can explain which view first stopped moving and what changed on the next run. That gives a teammate enough information to repeat the test without watching a whole stream.

AI helped draft and localize this article. The timestamp comparison is a reproducible calculation.