Dev.to WebDev 🛠 Dev 👁 0 📖 6 min read

Create an Interactive Audio Story Generator

Introduction Imagine a web app that lets users type a few prompts, choose a character’s voice, and instantly hear a fully‑voiced short story. With modern text‑to‑speech (TTS) APIs, that’s no longer a futuristic demo—it

Introduction

Imagine a web app that lets users type a few prompts, choose a character’s voice, and instantly hear a fully‑voiced short story. With modern text‑to‑speech (TTS) APIs, that’s no longer a futuristic demo—it’s something you can build in a single afternoon. In this post we’ll walk through an interactive audio story generator using Python for the backend, a tiny JavaScript front‑end, and the powerful TTS platform from ElevenLabs.

You’ll learn how to:

  • Call the ElevenLabs API to synthesize natural‑sounding speech.
  • Swap between multiple voices or even clone a custom voice.
  • Wire up a simple “choose‑your‑own‑adventure” loop that reacts to user choices.
  • Serve the audio files back to the browser for immediate playback.

Let’s dive in!

Prerequisites

What you need Why it matters
Python 3.9+ To run the server and make HTTP calls.
FastAPI (or Flask) A lightweight web framework that plays nicely with async requests.
Node.js (optional) For the front‑end dev server if you prefer npm run dev.
ElevenLabs API key You’ll need an API key from the ElevenLabs platform. Sign up at the affiliate link above to get one instantly.
A modern browser To test the audio playback.

If you’re comfortable with Docker, you can containerize the whole thing, but it’s not required for the core concepts.

Setting Up the ElevenLabs API

ElevenLabs offers a straightforward REST endpoint. After you obtain your API key, store it in an environment variable so you don’t accidentally commit it:

export ELEVENLABS_API_KEY="your_api_key_here"

A quick curl test proves everything works:

curl -X POST "https://api.elevenlabs.io/v1/text-to-speech/EXAMPLE_VOICE_ID" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "text": "Hello, world! This is a test of ElevenLabs TTS.",
        "model_id": "eleven_monolingual_v1"
      }' \
  --output hello.wav

If you hear a crisp hello.wav file, you’re ready for the next step. Replace EXAMPLE_VOICE_ID with any voice ID you see in the ElevenLabs dashboard (e.g., 21m00Tcm4TlvDq8ikWJ).

Generating Speech from Python

Below is a minimal FastAPI endpoint that receives a story fragment, calls ElevenLabs, and returns a temporary URL to the generated audio.

# app.py
import os
import uuid
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import httpx

app = FastAPI()
ELEVEN_API_KEY = os.getenv("ELEVENLABS_API_KEY")
BASE_URL = "https://api.elevenlabs.io/v1"

class StoryChunk(BaseModel):
    text: str
    voice_id: str = "21m00Tcm4TlvDq8ikWJ"   # default voice

@app.post("/speak")
async def speak(chunk: StoryChunk):
    endpoint = f"{BASE_URL}/text-to-speech/{chunk.voice_id}"
    payload = {
        "text": chunk.text,
        "model_id": "eleven_monolingual_v1"
    }
    headers = {
        "xi-api-key": ELEVEN_API_KEY,
        "Content-Type": "application/json"
    }

    async with httpx.AsyncClient() as client:
        resp = await client.post(endpoint, json=payload, headers=headers)
        if resp.status_code != 200:
            raise HTTPException(status_code=resp.status_code,
                                detail="ElevenLabs synthesis failed")
        # Save to a temp file
        file_id = f"{uuid.uuid4()}.mp3"
        with open(f"/tmp/{file_id}", "wb") as f:
            f.write(resp.content)

    return {"audio_url": f"/audio/{file_id}"}

@app.get("/audio/{file_id}")
async def get_audio(file_id: str):
    file_path = f"/tmp/{file_id}"
    if not os.path.exists(file_path):
        raise HTTPException(status_code=404, detail="Audio not found")
    return fastapi.responses.FileResponse(file_path, media_type="audio/mpeg")

Run the server:

uvicorn app:app --reload

Now any client can POST JSON like { "text": "Once upon a time..." } and receive a URL that streams the spoken snippet.

Building the Interactive Loop (Front‑End)

The front‑end is a single HTML page with a tiny JavaScript controller. It lets the user type a prompt, choose a voice, and then displays the next set of choices based on the story’s state.

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Audio Story Generator</title>
  <style>
    body { font-family: sans-serif; max-width: 600px; margin: 2rem auto; }
    #choices button { margin: 0.5rem; }
  </style>
</head>
<body>
  <h1>Interactive Audio Story</h1>
  <textarea id="prompt" rows="3" placeholder="Write your opening line..."></textarea><br>
  <select id="voice">
    <option value="21m00Tcm4TlvDq8ikWJ">Narrator (default)</option>
    <option value="EXAMPLE_VOICE_2">Mysterious Voice</option>
  </select>
  <button id="startBtn">Start</button>

  <div id="player"></div>
  <div id="choices"></div>

  <script>
    const startBtn = document.getElementById('startBtn');
    const prompt = document.getElementById('prompt');
    const voice = document.getElementById('voice');
    const playerDiv = document.getElementById('player');
    const choicesDiv = document.getElementById('choices');

    async function speak(text) {
      const resp = await fetch('/speak', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ text, voice_id: voice.value })
      });
      const data = await resp.json();
      const audio = new Audio(data.audio_url);
      playerDiv.innerHTML = '';
      playerDiv.appendChild(audio);
      audio.play();
    }

    // Simple story engine – in reality you’d store state on the server
    const storyMap = {
      "start": ["You stand at a crossroads. Go left or right?"],
      "left": ["A dragon appears! Fight or flee?"],
      "right": ["You find a treasure chest. Open it or ignore?"]
    };

    function presentChoices(key) {
      choicesDiv.innerHTML = '';
      const options = key === 'start' ? ['left', 'right'] :
                      key === 'left' ? ['fight', 'flee'] :
                      ['open', 'ignore'];
      options.forEach(opt => {
        const btn = document.createElement('button');
        btn.textContent = opt;
        btn.onclick = async () => {
          await speak(storyMap[opt] ? storyMap[opt][0] : `You chose ${opt}.`);
          presentChoices(opt);
        };
        choicesDiv.appendChild(btn);
      });
    }

    startBtn.onclick = async () => {
      await speak(prompt.value || "Once upon a time...");
      presentChoices('start');
    };
  </script>
</body>
</html>

What’s happening?

  1. The user writes an opening line (or uses the default).
  2. The front‑end POSTs that line to /speak.
  3. The server returns a temporary MP3 URL; the browser plays it.
  4. A very simple state machine (storyMap) decides the next set of options.
  5. Each button press triggers another call to /speak, creating a seamless audio loop.

Feel free to replace storyMap with a more sophisticated branching narrative stored in a database or a JSON file.

Optional: Voice Cloning for Personalized Characters

ElevenLabs also lets you clone a voice from a short audio sample (usually ~30 seconds). This is perfect for giving each story character a unique timbre.

# clone_voice.py
import os, httpx

API_KEY = os.getenv("ELEVENLABS_API_KEY")
BASE = "https://api.elevenlabs.io/v1"

def clone(name: str, audio_path: str):
    with open(audio_path, "rb") as f:
        files = {"audio_file": (audio_path, f, "audio/wav")}
        data = {"name": name}
        headers = {"xi-api-key": API_KEY}
        resp = httpx.post(f"{BASE}/cloning", headers=headers, data=data, files=files)
        resp.raise_for_status()
        return resp.json()["voice_id"]

voice_id = clone("Hero", "hero_sample.wav")
print("New voice ID:", voice_id)

After cloning, you can drop the new voice_id into the front‑end’s <select> list, letting users hear their custom character speak. The cloning endpoint is part of the ElevenLabs suite, so you get the same high‑quality neural models you already love.

Putting It All Together

  1. Spin up the FastAPI server (uvicorn app:app --reload).
  2. Serve index.html from the same directory or via a static file server (python -m http.server works fine).
  3. Open the page in Chrome/Firefox. Type an opening line, pick a voice, and watch the story unfold—audio‑first.
  4. Iterate: add more branches, store user choices in a DB, or integrate with a ChatGPT‑style text generator to make the story truly dynamic.

That’s it! You now have a functional, voice‑driven interactive story generator that you can extend into games, language‑learning tools, or accessibility experiences.

Next Steps

  • Persist stories – use SQLite or MongoDB to keep track of each user’s path.
  • Add AI text generation – combine OpenAI’s GPT‑4 with ElevenLabs to generate on‑the‑fly story content.
  • Deploy – containerize with Docker, push to Render, Fly.io, or any cloud provider.
  • Explore more voices – ElevenLabs offers a growing library of expressive voices; experiment with tone, speed, and emotion.

Call to Action

Ready to give your projects a natural‑sounding voice? Head over to ElevenLabs, grab an API key, and start building today. The platform’s generous free tier makes it easy to prototype, and the cloning feature lets you add truly unique characters to any interactive experience. Happy coding!

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.

Share 𝕏 Twitter ✈ Telegram ↗ Reddit 🟧 HN
← News 🛠 Dev 🏠 Home