Building a browser mashup maker: stem split, tempo stretch, key shift and drum alignment in Web Audio

I build SongUp AI. This is the story of rebuilding one of its free tools, the Mashup Creator, and what I learned doing it.

The bug report was one word: "quality"

Our first mashup tool was really a DJ crossfader. Song A played, then both songs overlapped at full volume for a while, then Song B played. Keys were never matched and the sum clipped. One user ran it nine times and then left with a one-word reason: "quality". They were right. That isn't a mashup.

What people mean by a mashup is simpler and harder at the same time: the vocals of one song over the music of another, sounding like they belong together. So the tool now has exactly two inputs: Vocals from and Beat from.

The pipeline

Everything runs in the browser. The songs are never uploaded.

1. Split both songs into stems. We run a 4-stem htdemucs model with ONNX Runtime Web (WebGPU when it's available, otherwise WASM) in a worker. Each song is capped at 200 seconds so a phone can finish the job. The model downloads once and is cached after that.
2. Find the tempo of each song from its drum stem, and fold half and double tempos so a 70 BPM ballad and a 140 BPM beat count as compatible.
3. Find the key. The first version used Krumhansl key profiles on the bass and "other" stems. That works for whole songs but picks the wrong mode surprisingly often. The current version compares each sung frame with the beat's chord at the same moment (after alignment) and picks the shift that fits best. Whole-song notes, then the classic key estimate, are the fallbacks.
4. Stretch and shift the vocals. The vocal stem is time-stretched to the beat's BPM and pitch-shifted to its key, or to a neighbouring key a fifth away when that's a smaller move. Small moves sound much more natural than big ones.
5. Line the vocals up on the beat. We cross-correlate the drum stem of the vocal song with the drum stem of the beat song, searching ±2 bars. If that correlation is weak, we snap the vocal to the beat grid instead.
6. Mix. A high-pass filter and a compressor go on the vocal so it sits at about the beat's level, with a small dip around 2.5 kHz in the beat to make room for the voice. A limiter goes on the master, peaking at −1 dBFS.
7. Hand control back. There are sliders for voice and beat level, key ±4 semitones and timing ±8 beats. You can download MP3 or WAV.

Things that surprised me

• Testing a 3-minute browser job is the hard part. Our dev server reloaded whenever another file in the checkout changed, which killed long runs. What worked was an esbuild bundle of just the audio module, served as a static page, posting results to a tiny local receiver.
• Don't add COOP/COEP headers to a test harness on a whim. They switched ONNX Runtime to multi-threaded mode, and in our setup that hung.
• Asking an AI to judge audio quality is noisy. We tried scoring single files with an LLM, and the same file got a 9 and then a 3. Comparing two files side by side, in both orders, gave usable rankings. Even then, the model tended to prefer whichever clip came second.
• A fit score helps. If the best key fit is low, the UI says so. Some pairs just don't work, and it's better to say so than to pretend they do.

Try it, and a note on rights

The tool is free with a free account, with unlimited MP3 and WAV downloads: Mashup Creator. If you want to compare it with other free options (Mixxx, Audacity, RaveDJ, Fadr), there's a side-by-side here: best free mashup makers.

Use tracks you made or have permission to use, especially if you plan to publish the result.

Questions about the DSP side are welcome in the comments.