Our mashup tool was just a crossfader. Rebuilding it taught me what a mashup really needs

I build SongUp AI, an AI song maker with a set of free audio tools that run in the browser. One of those tools was called a Mashup Creator, and for a while it did not make mashups.

The first version played song A, overlapped both full songs at full volume for up to a minute, then played song B. Keys were never matched, and the sum clipped. A paying user ran it nine times and cancelled the same day, with one word as the reason: quality. They were right.

Here is what the rebuild looks like, and the parts that turned out to matter.

A mashup is vocals over a beat, not two songs at once

The real job is narrow: take the singer from one song and put them on the instrumental of another. That needs four things in order, and each one depends on the last.

1. Split both songs

Both tracks go through an htdemucs 4-stem model. Vocals come from one song, and everything except vocals comes from the other. On a computer this runs in the browser, so the audio is not uploaded. The cost is a 172 MB model download on the first run. Each song is capped at 200 seconds to keep the run time sane.

Keep the stems you do not use in the final mix. They come back in steps 3 and 4.

2. Match the tempo, but fold it first

If the vocal song is at 70 BPM and the beat is at 140, you do not want to stretch by 2x. Half and double time are folded before any stretching, so the vocal moves as little as possible.

3. Match the key against the chords, not the whole song

This one surprised me. My first approach detected a key for each song from the bass and "other" stems using Krumhansl profiles, then shifted the vocal to match. It works, but pop songs often sit between two keys, so the label can be misleading.

What worked better is local: after alignment, compare each sung frame with the beat's chord at that moment, and pick the shift with the best fit across the song. Whole-song note matching and key labels stay as fallbacks. In pairwise listening comparisons the local method came out at least as good as whole-song notes, and both beat the plain key match.

4. Line up the vocals using the drums

The vocal stem has no reliable beat in it, because a voice does not sit on a grid. But the song it came from has a drum stem. So the tool correlates the vocal song's drums with the beat's drums, within two bars either way. That gives you the offset, and the offset applies to the vocal. If correlation fails, the vocals snap to the beat grid instead.

The mix is small, and it matters

• High-pass and compress the vocal so it sits at about the level of the beat.
• Dip the beat around 2.5 kHz to leave room for the voice.
• Limit the master to -1 dBFS.

Three steps. They are the difference between a voice on top of a track and a voice inside it.

Things I would tell myself earlier

• Leave the listener some controls. Voice and beat level, key shift of ±4 and timing of ±8 beats fix most of what automation misses.
• Do not trust a single AI score for audio. When I asked a model to rate files one at a time, the same file got a 9 and then a 3. Compare two clips, in both orders.
• Phones are a different product. A phone cannot run this in the browser, so phones and tablets split on the server, which needs a paid plan. That path needed about five minutes in the foreground, and one run failed for a user with no way to resume. Pending jobs now live in localStorage and pick up on the next open.

Try it

A 27-second look at it:

The tool is free with a free account on a computer, with no watermark, and it downloads MP3 or WAV. It takes audio files only, not YouTube or Spotify links. If you publish a mashup of commercial songs, you need permission from the rights holders.

https://www.songupai.com/tools/mashup-creator