A mock interviewer that speaks, listens, and proctors with no AI bill

Demo: https://drive.google.com/file/d/13WEiJXqwq7O6XzouhUR0LpNfgz67znd9/view?usp=sharing
Mentors on my platform wanted to run practice interviews with their mentees: a voice
asks a question, the candidate answers out loud, it gets recorded, and — because it's
practice for the real thing the session is proctored. Every off-the-shelf version of
this hands you three metered bills: a TTS service to read the question, an STT
service (usually Whisper) to transcribe the answer, and a proctoring vendor to watch the
room. Three subscriptions for a feature people use in short bursts.

The naive plan is to just pay them. Wire up a cloud TTS voice, POST each audio answer to
a speech-to-text endpoint, embed a proctoring SDK, and eat the per-minute cost forever.
It works, but every one of those is a recurring charge, a data-egress path, and a privacy
footprint — for something that's fundamentally the candidate's own browser talking to
the candidate's own microphone. I didn't want a network round-trip sitting between a
mentee and a practice question.

The decision

I asked how much of the stack the browser already ships, and the answer was: nearly all
of it. The whole "AI interviewer" experience is three native Web APIs in a trench coat.

TTS — the interviewer speaks. window.speechSynthesis renders the question on-device.
The mentor even tunes the pitch/rate so the interviewer ("Aria") has a personality. No
audio is generated on a server; nothing is downloaded.

STT — the answer transcribes itself. SpeechRecognition (webkitSpeechRecognition
in Chromium) does live, streaming transcription on-device — interimResults means
words land on screen as they're spoken, the same engine behind your phone keyboard's mic
button. No upload, no Whisper invoice.

Audio streaming — the recording is the ground truth. MediaRecorder captures the mic
to a chunked audio/webm blob. That blob is the only byte that touches my server, and
it goes to storage I already pay for.

// TTS: the interviewer reads the prompt (mentor-tuned voice)
const u = new SpeechSynthesisUtterance(prompt);
u.rate = interviewer.rate; u.pitch = interviewer.pitch;
speechSynthesis.cancel();            // never overlap two questions
speechSynthesis.speak(u);

// STT: live, streaming transcript — no server in the loop
const rec = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
rec.continuous = true; rec.interimResults = true;
rec.onresult = (e) => setTranscript([...e.results].map(r => r[0].transcript).join(''));

// Audio streaming: MediaRecorder is the ground truth if STT is unsupported
const mr = new MediaRecorder(stream, { mimeType: 'audio/webm' });
mr.ondataavailable = (e) => e.data.size && chunks.push(e.data);

The key judgment call is layering, not just "use the free thing." STT support is uneven
(Firefox notably lags), so transcription is never the source of truth — the recording
is. STT degrades to a nicety; the feature never breaks. And a green-room mic check runs a
lightweight VAD-style level meter off the Web Audio AnalyserNode (RMS on the time-
domain data) so a candidate can see the mic is live before the clock starts — again,
zero network, pure getByteTimeDomainData.

Tally: TTS $0, STT $0, proctoring $0. The only recurring cost is the audio blob storage I
already had.

The gotcha

Proctoring is where "free browser signals" nearly bit me. The harness itself is cheap and
honest — fullscreen lock, tab/blur focus-loss counting, copy/paste/right-click blocking,
periodic webcam snapshots, all buffered client-side and flushed in batches. The trap was a
kit option I was quietly proud of: "require camera."

I shipped it, then during testing found the guarantee was hollow. A candidate could allow
the camera at the start, then turn it off mid-interview from the browser's site
controls. The session sailed on — and worse, my snapshot loop kept firing, dutifully
uploading black frames. On the surface everything looked compliant. "Require camera"
that you can switch off after ten seconds isn't a requirement; it's decoration.

The fix is one signal: a MediaStreamTrack's health. When a camera is turned off, covered,
or unplugged, its track goes muted/ended and readyState stops being 'live'. Poll
that, and two behaviors fall out of the same check.

const track = (video.srcObject as MediaStream)?.getVideoTracks?.()[0];
const live = !!track && track.readyState === 'live' && track.enabled && !track.muted;

// (1) block instead of silently continuing
if (!live) { setCameraLive(false); log('camera_off'); }   // → full-screen "turn it back on" overlay

// (2) never upload a black frame from a dead camera
if (!track || track.readyState !== 'live' || track.muted) return; // skip the snapshot

Now when the camera drops, the runner throws a full-screen block — "your camera is off,
turn it back on to continue" with a one-click Retry camera — and the candidate can't
answer or advance until it's back. The clock keeps running and the off→on gap is logged,
so turning the camera off to buy thinking time is a recorded act, not a free one. And the
mentor's proctor gallery gets a clean "Camera turned off" flag instead of a wall of
black rectangles. Same track state, both jobs.

What I'd do next

• STT confidence + a manual-correction pass. On-device recognition trails Whisper on accents and noise. I'd surface a confidence score and let the candidate fix the transcript, keeping the audio as the authority.
• Real VAD, not just a level meter. The RMS reading is enough for a "mic is live" cue; proper voice-activity detection could auto-trim dead air and auto-advance on silence.
• Honest limits on proctoring. Client-side signals raise the cost of cheating; they don't make it impossible (a second device still exists). The goal is deterrence and an auditable timeline for a mentor — not a courtroom — and the UI should say so.

The meta-lesson: before you add an AI line-item, check what window already does for free.
The browser quietly grew a whole speech-and-media stack — TTS, streaming STT, MediaRecorder,
Web Audio VAD, MediaStreamTrack health — and most of us keep reaching for an API instead.

What's the last feature you paid a vendor for that the platform could already do TTS,
STT, or something else?