Create an Interactive Audio Story Generator
What’s an Interactive Audio Story Generator? Imagine you can hand‑write a short story, then let your code turn it into a full‑blown audiobook with multiple characters, sound effects, and even user‑controlled pacing. Th
What’s an Interactive Audio Story Generator?
Imagine you can hand‑write a short story, then let your code turn it into a full‑blown audiobook with multiple characters, sound effects, and even user‑controlled pacing. That’s the goal of an Interactive Audio Story Generator. It blends natural language processing, text‑to‑speech (TTS), and voice‑cloning to create immersive audio experiences that feel like a real conversation.
Below I’ll walk you through building a lightweight prototype in Python, show how to hook up a voice‑cloning service, and give you a few tricks for making the story interactive. If you’re new to voice AI, you’ll see why ElevenLabs is a great starting point.
1. Why TTS and Voice Cloning Matter
- TTS turns plain text into speech. Modern models can generate speech that sounds natural, with expressive intonation, and can be fine‑tuned to specific accents or emotions.
- Voice cloning lets you emulate a real person’s voice (or create a custom synthetic voice). That means each character in your story can have a distinct timbre, making the audio feel like a true narrative.
- Interactivity can be added by letting the user choose plot twists, adjust speed, or even answer questions that influence the story flow.
Combining these pieces gives you a “choose‑your‑own‑adventure” experience entirely in audio form.
2. Tooling Overview
| Component | Why it’s useful | Example |
|---|---|---|
| Python | Rapid prototyping, rich libraries |
requests, pydub
|
| ElevenLabs API | State‑of‑the‑art TTS & voice cloning | https://try.elevenlabs.io/kr07zfuqn1bp |
| Flask | Simple web interface to trigger stories |
/play endpoint |
| Web Audio API (in the browser) | Control playback, speed, volume | AudioContext |
We’ll focus on the Python side and show how to integrate ElevenLabs. If you’re more comfortable with JavaScript or curl, the same concepts apply.
3. Setting Up ElevenLabs
- Sign up at the affiliate link:
https://try.elevenlabs.io/kr07zfuqn1bp - Grab your API key from the dashboard.
- (Optional) Create a custom voice by uploading a few minutes of clean audio. ElevenLabs will generate a voice model you can reference by ID.
4. Project Skeleton
audio-story/
├─ app.py
├─ requirements.txt
├─ story.json
└─ static/
└─ index.html
-
story.jsonholds the narrative structure. -
app.pyhandles the API calls. -
static/index.htmlis a minimal UI.
5. Sample Story JSON
{
"title": "The Lost Lighthouse",
"characters": {
"narrator": "narrator",
"old_man": "old_man",
"young_woman": "young_woman"
},
"script": [
{
"speaker": "narrator",
"text": "It was a stormy night when the lighthouse keeper, old Man, decided to leave the shore."
},
{
"speaker": "old_man",
"text": "I must check the lantern. The sea will not forgive neglect."
},
{
"speaker": "young_woman",
"text": "Hold on, old man! I’ve heard the sea is restless tonight."
}
]
}
Feel free to expand this with branching logic or user prompts.
6. Python: Calling ElevenLabs
First, install the dependencies:
pip install requests Flask pydub
6.1. Basic TTS Function
import requests
ELEVENLABS_API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.elevenlabs.io/v1"
def synthesize_text(text, voice_id, output_path="output.mp3"):
headers = {
"xi-api-key": ELEVENLABS_API_KEY,
"Content-Type": "application/json"
}
payload = {
"text": text,
"voice_id": voice_id,
"model_id": "eleven_monolingual_v1"
}
resp = requests.post(f"{BASE_URL}/text-to-speech/{voice_id}", json=payload, headers=headers)
resp.raise_for_status()
with open(output_path, "wb") as f:
f.write(resp.content)
print(f"Saved to {output_path}")
Replace voice_id with the ID of the narrator or any cloned voice.
6.2. Batch Script Generation
import json
from pathlib import Path
def generate_story_audio(story_path, output_dir="audio"):
Path(output_dir).mkdir(exist_ok=True)
with open(story_path) as f:
story = json.load(f)
for idx, line in enumerate(story["script"], 1):
speaker = line["speaker"]
voice_id = story["characters"][speaker] # map to ElevenLabs voice ID
text = line["text"]
output_file = Path(output_dir) / f"{idx:02d}_{speaker}.mp3"
synthesize_text(text, voice_id, str(output_file))
Now you can call:
generate_story_audio("story.json")
and you’ll get a folder full of MP3s, each representing a line.
7. Making It Interactive
7.1. Speed Control
ElevenLabs supports speed_scale (0.5–2.0). Add it to the payload:
payload = {
"text": text,
"voice_id": voice_id,
"model_id": "eleven_monolingual_v1",
"voice_settings": {
"stability": 0.5,
"similarity_boost": 0.75,
"style": "default",
"style_degree": 0.5,
"speed_scale": 1.2 # user‑controlled
}
}
7.2. Branching Logic
Add a choices field in your JSON:
{
"speaker": "young_woman",
"text": "Do you want to stay or leave?",
"choices": [
{"label": "Stay", "next": 5},
{"label": "Leave", "next": 7}
]
}
Your Flask route can parse the user’s choice and jump to the specified index.
8. Frontend Hook (Optional)
<!-- static/index.html -->
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Interactive Audio Story</title>
</head>
<body>
<h1>Play the Story</h1>
<button id="play">Play</button>
<script>
const playBtn = document.getElementById('play');
playBtn.onclick = async () => {
const response = await fetch('/play');
const audioUrl = await response.text();
const audio = new Audio(audioUrl);
audio.play();
};
</script>
</body>
</html>
And in app.py:
from flask import Flask, send_file
app = Flask(__name__)
@app.route('/play')
def play():
# Assume we have concatenated MP3s into one file
return send_file('full_story.mp3', mimetype='audio/mpeg')
9. Production Tips
| Tip | Why |
|---|---|
| Cache audio | Re‑generating TTS for each request is slow and costly. Store MP3s in S3 or local disk. |
| Use async | For high‑traffic apps, use aiohttp or FastAPI to handle many concurrent requests. |
| Secure API keys | Never expose your ElevenLabs key in client code. Store it in environment variables or a secrets manager. |
| Add fallback | If the TTS API fails, have a local TTS fallback (e.g., pyttsx3) so your app stays responsive. |
10. Wrap‑Up
You’ve now seen how to:
- Pull in ElevenLabs for high‑quality TTS and voice cloning.
- Structure a story with multiple characters.
- Add simple interactivity (speed changes, branching).
- Build a lightweight web interface to play the audio.
The real fun begins when you start adding sound effects, background music, or even real‑time user input. The possibilities are endless.
Call to Action
If you’re ready to bring your audio stories to life, start by signing up at ElevenLabs through this link:
https://try.elevenlabs.io/kr07zfuqn1bp
They offer a generous free tier and a powerful API that will make your interactive audio generator feel like a polished product. Happy coding—and may your stories be loud, clear, and unforgettable!
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.