Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 1 min read

Why Turn-Based Voice AI Feels Broken: Building a 52ms Barge-In Loop (HardCall)

Most "voice AI" demos feel unnatural for one simple reason: they are still turn-based walkie-talkies under the hood. When a user interrupts the AI mid-sentence, a typical wrapper waits for endpointing (300ms+), waits fo

Why Turn-Based Voice AI Feels Broken: Building a 52ms Barge-In Loop (HardCall)

Most "voice AI" demos feel unnatural for one simple reason: they are still turn-based walkie-talkies under the hood.

When a user interrupts the AI mid-sentence, a typical wrapper waits for endpointing (300ms+), waits for STT finalization (200ms+), and then cancels the server-side TTS stream while WebSocket frames are still draining. The result is 650ms–900ms of awkward cross-talk.

While building HardCall β€” an open-source full-duplex AI voice interview & high-stakes drill simulator live at hardcall.statementflow.win β€” we replaced turn-based waiting with a 52ms dual-loop barge-in architecture.

HardCall 52ms Dual-Loop Barge-In Architecture

1. Never Wait for STT End-of-Turn

If you wait for a transcript before cutting audio, you have already lost the human conversation feel. Humans notice interruption lag above ~200ms.

HardCall splits interruption into two asynchronous loops:

  1. 50ms Audio Fast-Loop (Deterministic): Local Silero VAD detects speech onset (~18ms) and immediately invokes to flush the outbound audio ring buffer in 0.16ms. Zero server round-trip required to stop talking.
  2. 200ms Semantic Slow-Loop (Intent Gate): While audio is already silent, a lightweight intent gate checks whether the user's utterance is a real rebuttal ("Waitβ€”our p99 dropped 42%...") or meaningless backchannel/noise ("uh-huh", cough).

2. Why Dual-Loop Beats Raw Energy Thresholds

Pure energy VAD cuts out on keyboard clicks or throat clearing. Pure LLM turn-detection feels sluggish.

By combining:

  • Instant local buffer flush on speech onset
  • State reset to drop stale TTS frames
  • Fast semantic classification to either pivot the rebuttal prompt (<200ms) or smoothly resume

you get crisp 52.16ms perceived barge-in without false-positive interruptions derailing the session.

3. Try It Live or Inspect the Source

If you are building real-time voice agents over WebRTC or raw 8kHz RTP, I'd love to hear how you handle outbound buffer flushing and false-trigger recovery!

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.