Create an AI Podcast Generator with ElevenLabs

Introduction

Podcasts have exploded in popularity, but producing a full episode still takes hours of recording, editing, and post‑production. What if you could feed a script into a service, let an AI voice narrate it, add a splash of music, and output a ready‑to‑publish MP3? In this article we’ll build a simple AI Podcast Generator that does exactly that, using the powerful text‑to‑speech (TTS) capabilities of ElevenLabs.

You’ll see how to:

• Call the ElevenLabs API from Python (or curl) to generate realistic speech.
• Combine multiple audio chunks into a single episode with pydub.
• Optionally layer background music and export a podcast‑ready file.

By the end you’ll have a reusable script you can plug into any content pipeline—whether you’re turning blog posts into audio, automating interview recaps, or just experimenting with voice cloning.

Getting Started

First, make sure you have Python 3.8+ installed. We’ll rely on a few lightweight packages:

pip install requests pydub tqdm

• requests – for HTTP calls to the ElevenLabs API.
• pydub – for audio concatenation and basic mixing.
• tqdm – optional, but handy for progress bars when processing long scripts.

You’ll also need ffmpeg on your system because pydub uses it under the hood. On macOS you can install it via Homebrew:

brew install ffmpeg

Linux and Windows have similar installers; just make sure ffmpeg is on your PATH.

Setting Up ElevenLabs

ElevenLabs offers a developer‑friendly REST API that can generate high‑fidelity speech from plain text. Sign up for a free account and grab your API key from the dashboard. When you’re ready, you can start using the endpoint:

POST https://api.elevenlabs.io/v1/text-to-speech/{voice_id}

For a quick start, use the default “Rachel” voice (voice_id = 21m00Tcm4TlvDq8ikWAM). If you want a custom cloned voice, upload a few minutes of reference audio through the ElevenLabs UI first—then replace the voice_id with your own.

Tip: The API is rate‑limited, so batch your requests or add a short time.sleep() between calls if you’re processing long scripts.

You can sign up and get your key here: https://try.elevenlabs.io/kr07zfuqn1bp

Generating Speech from Text

Below is a minimal Python function that sends a chunk of text to ElevenLabs and returns an AudioSegment object ready for further processing.

import os
import requests
from pydub import AudioSegment
from io import BytesIO

ELEVEN_API_KEY = os.getenv("ELEVEN_API_KEY")
VOICE_ID = "21m00Tcm4TlvDq8ikWAM"   # default voice; replace with your own if needed
BASE_URL = f"https://api.elevenlabs.io/v1/text-to-speech/{VOICE_ID}"

def synthesize(text: str) -> AudioSegment:
    """Call ElevenLabs TTS and return a pydub AudioSegment."""
    headers = {
        "xi-api-key": ELEVEN_API_KEY,
        "Content-Type": "application/json"
    }
    payload = {
        "text": text,
        "model_id": "eleven_monolingual_v1",  # the default high‑quality model
        "voice_settings": {
            "stability": 0.75,
            "similarity_boost": 0.85
        }
    }

    response = requests.post(BASE_URL, json=payload, headers=headers)
    response.raise_for_status()
    audio_bytes = BytesIO(response.content)
    return AudioSegment.from_file(audio_bytes, format="mp3")

Splitting Long Scripts

ElevenLabs imposes a ~5 KB limit per request (roughly 300–400 words). To handle longer episodes, split the script into manageable paragraphs:

from tqdm import tqdm

def script_to_segments(script: str) -> list[AudioSegment]:
    paragraphs = [p.strip() for p in script.split("\n\n") if p.strip()]
    segments = []
    for para in tqdm(paragraphs, desc="Synthesizing"):
        seg = synthesize(para)
        segments.append(seg)
    return segments

Stitching Episodes Together

Once you have a list of AudioSegments, concatenating them is straightforward:

def assemble_episode(segments: list[AudioSegment]) -> AudioSegment:
    episode = AudioSegment.empty()
    for seg in segments:
        # Add a short 300 ms pause between paragraphs for natural pacing
        episode += seg + AudioSegment.silent(duration=300)
    return episode

Export the final episode as an MP3 ready for any podcast host:

episode = assemble_episode(script_to_segments(my_script))
episode.export("my_podcast_episode.mp3", format="mp3", bitrate="192k")
print("✅ Episode saved as my_podcast_episode.mp3")

Optional: Adding Background Music

A subtle music bed can make your podcast feel more professional. Here’s a quick way to layer a looped music track under the narration:

def add_background(episode: AudioSegment, music_path: str, volume_reduction: int = -20) -> AudioSegment:
    music = AudioSegment.from_file(music_path).apply_gain(volume_reduction)

    # Loop music to cover the entire episode length
    repeats = int(episode.duration_seconds // music.duration_seconds) + 1
    music_loop = music * repeats
    music_loop = music_loop[:len(episode)]  # trim to exact length

    # Mix the two tracks (voice on top)
    return episode.overlay(music_loop)

episode_with_music = add_background(episode, "ambient_loop.mp3")
episode_with_music.export("my_podcast_with_music.mp3", format="mp3")

Feel free to experiment with fade‑ins/outs or different volume levels to suit your style.

Deploying Your Podcast Generator

If you want to turn this into a web service, a few options are:

• FastAPI – expose a /generate endpoint that accepts raw text and returns a downloadable MP3.
• Serverless (Vercel / AWS Lambda) – keep the runtime lightweight; just make sure the function has access to ffmpeg.
• GitHub Actions – schedule a daily run that pulls the latest blog post, converts it, and pushes the MP3 to an S3 bucket.

A minimal FastAPI route might look like this:

from fastapi import FastAPI, HTTPException
from fastapi.responses import FileResponse

app = FastAPI()

@app.post("/generate")
def generate_podcast(text: str):
    try:
        segs = script_to_segments(text)
        episode = assemble_episode(segs)
        out_path = "temp_episode.mp3"
        episode.export(out_path, format="mp3")
        return FileResponse(out_path, media_type="audio/mpeg", filename="episode.mp3")
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Deploy with uvicorn main:app --host 0.0.0.0 --port $PORT and you have an on‑demand podcast generator.

Wrap‑Up

You now have a fully functional pipeline:

1. Split your script into bite‑size chunks.
2. Send each chunk to ElevenLabs for ultra‑realistic voice synthesis.
3. Combine the resulting audio, optionally mix in background music.
4. Export a polished MP3 ready for publishing.

Because the ElevenLabs API returns high‑quality audio in a single request, the resulting voice sounds far more natural than traditional TTS engines. This opens the door to many creative applications—automated news briefs, multilingual podcasts, or even personalized audiobooks.

Ready to Give It a Spin?

All the heavy lifting is done by ElevenLabs, and you can start for free using the link below. Grab your API key, plug it into the script, and let the AI do the talking.

Try ElevenLabs now: https://try.elevenlabs.io/kr07zfuqn1bp

Happy coding, and may your podcasts be ever‑engaging!