Fish Audio Live 3 Claims 186 ms Full-Duplex Reactions — Dual-Track WASAPI Still Sets the Real Floor

On 10 November 2026 Fish Audio shipped Live 3, a full-duplex voice model aimed at agents. It keeps listening while it speaks, so a pause, a quick “mm-hmm”, or a mid-sentence correction can change the next output. The company says it can decide to emit a short reaction in as little as 186 ms. Turn-taking is learned inside the model from real conversations rather than bolted on as VAD thresholds and silence timers. English only at launch, available on their agent platform, API, and on-prem for enterprises.

Those numbers look good once clean PCM is already at the model. They do not change the capture budget on a Windows desktop when you need a private translation overlay for an interview or support call.

What a full-duplex cloud model still cannot see

Live 3 (and every other hosted duplex or streaming model) receives the stream you send it. It does not sit on your local audio endpoints. It does not give you a second, private track of your own microphone that never enters the call. It does not let you paint a translation overlay that is invisible to screen share. And it does not run when you prefer not to send audio off-box at all, or when the platform you are on has not been integrated.

For a developer who needs real-time speech-to-text and translation during a technical interview, a sales call, or a support session in another language, those constraints matter. The useful path is still local capture of two tracks, local or BYOK processing, and an overlay that never appears in the shared desktop.

Multiple waveforms. Dual-track capture keeps the mic and the remote side separate so each can be biased, timed, and translated independently.

Dual-track capture is still the practical floor

A single mixed stream loses speaker identity. For live translation you need:

• Mic track — your voice (private, never sent to the remote side unless you choose to)
• Loopback track — everything the other person says (WASAPI loopback of the render device)

That separation lets the STT engine produce clean transcripts for each side and the UI show them side-by-side without injecting a bot into the meeting.

On modern Windows the practical stack is:

1. Endpoint device (USB headset, Bluetooth HFP/A2DP, built-in mic, or WASAPI loopback).
2. Windows Audio Engine (audiodg.exe + audioeng.dll) in shared mode: mixing, any “communications” enhancements, spatial audio, vendor APOs. This routinely adds 10–40+ ms and is fragile under load or with Bluetooth.
3. Your capture client buffer (10–40 ms is the common practical range; lower values drop under DPC spikes or heavy system load).
4. Format conversion and ring-buffer copy off the audio callback (float32 → int16, 48 kHz → 16 kHz).
5. VAD or simple energy gate.
6. STT (local faster-whisper / Whisper.cpp / Qwen3-ASR GGUF, or a BYOK cloud endpoint) + optional translation.
7. Overlay paint on a dedicated thread (<16 ms if you stay off the UI thread).

A “186 ms reaction” cloud model is real once it has clean PCM. It is almost irrelevant if the samples have already been delayed or dropped inside the audio engine. End-to-end partials on a good day land in the 80–150 ms range; Bluetooth or a busy shared-mode session pushes that higher.

Minimal capture sketch

// Pseudocode shape of a dual-track WASAPI client
// Real code uses the Windows Audio Session API or a thin wrapper

struct DualCapture {
    mic: WasapiCapture,      // exclusive or shared, low-latency
    loopback: WasapiCapture, // render-device loopback
}

impl DualCapture {
    fn start(&mut self, period_ms: u32) {
        // Request smallest stable period the driver will honor
        self.mic.start(period_ms);
        self.loopback.start(period_ms);
        // Callback or event-driven pull into lock-free rings
    }

    fn pull_frames(&mut self) -> Option<(Vec<i16>, Vec<i16>)> {
        let mic = self.mic.read_converted(16000, 1)?;
        let remote = self.loopback.read_converted(16000, 1)?;
        Some((mic, remote))
    }
}

Keep the audio callback tiny. Do the resampling and any light VAD off-thread. Feed the two streams independently into your STT path so you can apply different biases, timeouts, or even different models per side.

Basic frequency and amplitude labels. Real capture graphs get messier once the Windows audio engine, Bluetooth, and vendor APOs enter the path.

Tradeoffs that actually show up in the wild

• Exclusive mode on the mic gives the lowest period but breaks other apps and some headsets. Shared mode is the pragmatic default for an overlay that must coexist with the call app.
• Bluetooth HFP is still a latency and quality tax. Prefer USB or the PC’s own mic + speakers when you can.
• Loopback of the render device captures whatever the call app is playing, including its own notifications. You still want a way to gate or bias that stream.
• Local models (Whisper.cpp, faster-whisper, small GGUF ASR) keep audio on-box and avoid network RTT. Cloud BYOK endpoints win on accuracy and language coverage when the link is clean.
• Overlay must be click-through and excluded from screen share. Paint on its own thread so a stalled UI never blocks the audio callback.

The 186 ms number is useful marketing once the samples have already left the machine. The floor you actually ship against is still the dual-track WASAPI path, the engine’s processing, and the buffer you choose.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.