Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 3 min read

I built a free AI clip maker that runs entirely in your browser (no uploads)

Most "AI clip maker" tools follow the same model: you upload a two-hour podcast to their servers, wait, and then pay in credits for every minute they process. I wanted to see how far I could get doing all of it in the br

Most "AI clip maker" tools follow the same model: you upload a two-hour podcast to their servers, wait, and then pay in credits for every minute they process. I wanted to see how far I could get doing all of it in the browser instead, so that the video never leaves the user's machine and the tool can be genuinely free.

The result is ClipTool, a free AI clip maker that takes a long video and gives you vertical, captioned clips for YouTube Shorts, TikTok and Reels. No uploads, no watermark, no credits. Here's how it works under the hood.

The pipeline

video file ──► extract audio (ffmpeg.wasm)
           ──► transcribe with word timestamps (Whisper, Web Worker)
           ──► score segments and pick the best moments
           ──► find the speaker's face (MediaPipe) ──► 9:16 crop path
           ──► burn in animated captions (libass) + optional CC0 music
           ──► MP4 clips, downloaded straight from memory

Every step runs on the client. The server only handles sign-in and serves static assets.

1. Audio extraction with ffmpeg.wasm

ffmpeg.wasm is FFmpeg compiled to WebAssembly. I pull a 16 kHz mono track out of the video, which is all the speech model needs:

await ffmpeg.exec(["-i", "input.mp4", "-ac", "1", "-ar", "16000", "-f", "f32le", "audio.raw"]);

Two lessons from production:

  • The multi-threaded build can hang on some machines. I default to the single-threaded core and wrap every exec in a timeout. It's slower but it never freezes the tab.
  • Multi-threading needs SharedArrayBuffer, which needs cross-origin isolation. Cross-Origin-Embedder-Policy: credentialless (instead of require-corp) lets third-party assets like fonts and model files keep loading without special headers.

2. Transcription: Whisper in a Web Worker

Speech-to-text uses transformers.js with a timestamped Whisper model (whisper-base_timestamped). It runs in a Web Worker so the UI stays responsive, and picks the fastest backend available:

  • WebGPU, when the browser supports it (fp32 encoder + q4 decoder)
  • WASM fallback (q8) everywhere else

Long videos are split into chunks and sent to the worker one at a time, each with its time offset, so word timestamps line up with the original video.

3. Picking the best moments

There's no LLM call here; it's a fast heuristic scorer, which keeps it free and offline-capable:

  1. Words are grouped into sentence-like segments at punctuation or pauses longer than ~0.7s.
  2. Candidate clips are built from every run of consecutive segments that fits the length range the user picks, so a clip always starts and ends on a natural cut point.
  3. Each candidate gets four scores:
    • Hook: does it open with a strong line? Hook words ("why", "the biggest mistake", "here's the reason"...), questions, numbers and direct "you" all score.
    • Payoff: does it end on a complete sentence, with strong points along the way?
    • Standalone: does it make sense without context? Opening on "and", "but" or "because" is penalised.
    • Emotion: energy and emphasis in the wording.
  4. overall = hookΒ·0.35 + payoffΒ·0.25 + standaloneΒ·0.25 + emotionΒ·0.15, then non-overlapping top picks win.

It's not magic, but it's surprisingly good at finding the moments a human editor would pick.

4. Auto-reframe to 9:16 with MediaPipe

Landscape video needs to become vertical without cutting the speaker's head off. I sample frames, run Google's MediaPipe face detector (BlazeFace short-range) in the browser, and turn the face positions into keyframes. Those become an FFmpeg crop expression that pans smoothly between keyframes instead of jumping.

5. Captions with libass

Captions are generated as an .ass subtitle file with word-level timing, so each word pops in as it's spoken, then burned in by FFmpeg's libass filter. Fonts are bundled (all SIL Open Font License: Poppins, Anton, Bebas Neue) so the output looks identical on every machine.

6. Copyright-free music

Background music comes from a curated library of CC0 tracks (public domain, no attribution needed), so clips never get claimed. You can browse the library here.

What I'd tell anyone building this

  • The browser is a real video workstation now. WebAssembly + WebGPU + Workers are enough for a full pipeline.
  • Design for the slowest device. Timeouts, single-thread fallbacks and chunking matter more than peak speed.
  • Privacy is a feature people notice. "Your video never leaves your device" is a promise you can only make if the work really happens on the device.

If you want to try it, it's free at freeclipmaker.com. I'd love feedback, especially on the clip-picking heuristics. What signals would you add?

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.