Dev.to AI 🤖 Ai Wed, 05 Aug 2026 22:05:14 +0000 👁 0 📖 2 min read

Building ALAD: Real-Time AI Voice Dubbing in Chrome with Gemini 3.5 & WebSockets

Have you ever watched a foreign video or live stream and wished you could hear it translated live in your own language instead of reading subtitles? To solve this, I built ALAD (Live Audio Dubbing) — an open-source Chro

Have you ever watched a foreign video or live stream and wished you could hear it translated live in your own language instead of reading subtitles?

To solve this, I built ALAD (Live Audio Dubbing) — an open-source Chrome extension that translates and dubs audio in real-time from any active Chrome tab into 78 languages using Google's Gemini 3.5 Live Translate API via bidirectional WebSockets.

In this post, I'll walk through how ALAD works under the hood, its architecture, and how you can try it out or contribute!

🚀 Key Features

Zero-Latency Audio Capture: Instantly captures tab audio using Chrome's native chrome.tabCapture API.
78 Languages Supported: Real-time voice dubbing into dozens of target languages.
Bidirectional WebSockets: Powered by Gemini 3.5 Live Translate (BidiGenerateContent) for real-time streaming audio input and output.
Usage & Analytics Dashboard: Built-in dashboard with activity heatmaps and hourly usage charts.
Privacy-First: Audio streams are processed locally and API keys remain stored securely on your device.

🛠️ Architecture Overview

The extension consists of three main components:


text
┌─────────────────────────┐      Audio Chunks       ┌──────────────────────────────┐
│                         ├────────────────────────►│                              │
│  Background Worker      │                         │  Gemini 3.5 Live Translate   │
│  (chrome.tabCapture)    │◄────────────────────────┤  (WebSocket API)             │
│                         │     Dubbed Audio        └──────────────────────────────┘
└───────────┬─────────────┘
            │
            ▼
┌─────────────────────────┐
│ Stats & Analytics       │
│ Dashboard               │
└─────────────────────────┘

Extension Popup & UI: Controls session state, target language selection, and API key configuration.

Background Service Worker (background.js): Captures tab audio streams via chrome.tabCapture, manages connection states, and logs session metadata.

Gemini Client (gemini.js): Establishes a bidirectional WebSocket connection to the Gemini Live Translate API (BidiGenerateContent), streaming PCM audio chunks and receiving translated speech in real time.

Stats Dashboard (/stats): Renders interactive activity heatmaps and usage charts.

💡 How It Works Under the Hood
1. Tab Audio Capture
When you click Start Dubbing, ALAD captures the media stream from the active tab:

JavaScript
chrome.tabCapture.capture({ audio: true, video: false }, (stream) => {
  if (!stream) {
    console.error("Failed to capture tab audio.");
    return;
  }
  // Process and stream PCM audio chunks
});

2. Real-Time Streaming over WebSockets
Raw audio PCM chunks are streamed continuously over a bidirectional WebSocket endpoint (BidiGenerateContent). This allows low-latency audio transmission and simultaneous receiving of dubbed audio fragments without waiting for complete sentences.

⚙️ Quick Start & Installation
Clone the repository:

Bash
git clone https://github.com/navidseyedain/ALAD.git
Open Chrome and navigate to chrome://extensions/.

Enable Developer Mode in the top-right corner.

Click Load unpacked and select the extension directory.

Enter your free Gemini API Key from Google AI Studio.

Choose your target language, click Start Dubbing, and play any video/audio in your tab.

🗺️ Roadmap & Contributing
ALAD is open-source under the MIT License. Future improvements include:

[ ] Chrome Web Store release

[ ] Automatic source language detection

[ ] Custom keyboard shortcuts

[ ] Per-language natural voice selection

Contributions and feedback are welcome! Feel free to open an issue or submit a pull request on GitHub.

🔗 GitHub Repository: navidseyedain/ALAD

If ALAD helps you enjoy foreign content without subtitles, please give the repo a ⭐️!

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.

← News 🤖 Ai 🏠 Home