How I Architected an Edge AI Pipeline for Podcast Tools

I started a podcast about indie development a couple of years ago. I absolutely loved the actual recording process—chatting with guests, diving deep into technical architectures, and sharing war stories. But the moment the recording stopped, my enthusiasm tanked.

I was struggling with the post-production admin. Staring at a blank text editor to write a catchy title, format show notes, and draft an SEO-friendly description felt like a massive chore. I would spend 45 minutes just trying to phrase the episode summary, completely draining the creative battery I needed for the next week's recording.

I realized that if I didn't automate this workflow, I was going to burn out and quit. So I decided to build a set of AI tools to handle the heavy lifting.

The Context-Switching Problem

When you're a solo creator or a small team, context switching is your biggest enemy. You go from a high-energy, conversational recording state directly into a low-energy, analytical writing state.

The cost of ignoring this friction is high. Inconsistent publishing schedules, listener drop-off due to poor discoverability, and eventually, podcast death. Podcast titles need to be compelling but not clickbait. Show notes need strict structure for accessibility and SEO. Doing this manually every single week is a recipe for frustration.

I needed a system that could take my raw, messy audio transcripts and instantly output structured, usable content without me having to micromanage the AI.

Technical Approach: Prompting and Edge Inference

Building these tools required more than just passing a transcript to an LLM and asking for "show notes." The output was usually a wall of unformatted text. I had to engineer the prompts and the architecture to enforce structure.

Here are the core practices I implemented when building the PodCrisp backend.

1. Constraining Title Generation

For the title generator, the biggest mistake developers make is giving the LLM too much freedom. If you just ask for "good titles," you get generic fluff. You need to enforce rules in the system prompt.

// Prompt engineering for podcast title generation
function buildTitlePrompt(topic, tone, count) {
  return {
    system: `You are an expert podcast producer. Generate compelling titles.
Rules:
- 5-10 words each
- Use curiosity gaps, numbers, or strong hooks
- Match the tone: ${tone}
- No clickbait that overpromises
- Format as numbered list`,
    user: `Episode topic: ${topic}

Generate ${count} title options that make someone want to click play.`
  };
}

// Example: generate 5 titles for a bootstrapping episode
const prompt = buildTitlePrompt(
  "How to start a podcast with zero budget",
  "inspirational",
  5
);

By explicitly defining the word count, forbidding overpromising clickbait, and setting a specific tone, the LLM's output shifts from generic suggestions to actual, usable hooks.

2. Structuring Show Notes at the Edge

For show notes, the challenge is taking a 5,000-word messy transcript and extracting signal from noise. I also wanted to ensure low latency, so I built the pipeline on Cloudflare Workers, running AI inference directly at the edge.

// Show notes generation pipeline (Cloudflare Workers + AI)
export default {
  async fetch(request, env) {
    const { transcript } = await request.json();

    // Call AI model for structured show notes
    const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: `Convert podcast transcripts to show notes:
## Episode Summary (2-3 sentences)
## Key Takeaways (5-7 bullet points)
## Timestamps (estimate from content flow)
## Resources Mentioned`
        },
        { role: 'user', content: transcript }
      ],
      max_tokens: 800,
    });

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' },
    });
  },
};

The secret here is the system prompt. By demanding specific Markdown headers and limiting the summary to 2-3 sentences, the model is forced to organize the data. Running this on Cloudflare Workers with edge AI inference means I avoid the cold-start penalties and latency spikes typical of centralized server setups.

Practical Results and Lessons Learned

What worked really well was the structured show notes pipeline. It consistently outputs clean Markdown that I can copy-paste directly into my podcast host. The title generator, when constrained by the prompt rules above, actually produces hooks that sound like a human producer wrote them.

What surprised me was the timestamp estimation. I initially asked the model to output exact MM:SS timestamps. It hallucinated them terribly. I had to adjust the prompt to say "estimate from content flow" and just use chapter markers instead of exact seconds. The LLM is great at narrative flow, but terrible at counting audio milliseconds.

If I were to change something today, I would implement more fine-tuned routing. A true-crime podcast needs a very different title tone than a B2B SaaS tutorial. I'm currently working on routing the prompts through different system personas based on the user's selected niche.

Try the Title Generator

If you're currently staring at a blank screen trying to name your next episode, I've extracted the title generation logic into a free, standalone tool. You can test out the prompt constraints and generate multiple style options without needing to sign up for anything.

You can try the free Podcast Title Generator here: https://podcrisp.com/free-tools/title-generator

Building these tools completely changed my relationship with podcasting. I actually look forward to the post-production phase now, which is a feeling I never thought I'd have.

Full disclosure: I'm the developer of PodCrisp. I built this because I needed it myself.