Designing a prompt builder for a text-to-music model: 500 characters, 4 instruments, and a lot of restraint

The input box is the product.

When the only interface to your model is a single text field, output quality is decided before generation ever starts. Feed a text-to-music model sad song and you get something generic — it has to guess the genre, the tempo, the instrumentation, whether you wanted vocals at all, and how long the track should be.

We build a text-to-song tool, and the failure mode we kept running into wasn't "the model is bad". It was "the prompt contained no decisions". So we built a prompt builder: a free, static page that turns a few clicks plus one short idea into a prompt the model can act on.

Here are the constraints we were working with, and the three decisions that followed.

The constraints come first

Before touching the UI, write down what the pipeline actually accepts. Ours:

The 500-character cap is the one that shapes everything else. It's generous for a specification and tiny for a wish. Every control you add to the UI eats budget, so the UI has to make decisions for the user instead of exposing raw parameters.

Decision 1: a controlled vocabulary, not free text

Rather than let people type adjectives, we defined a fixed vocabulary per dimension: 14 genres, 9 moods, 6 tempo bands, 16 instruments, 5 vocal configurations, 2 lyric languages, 6 intended uses, 4 durations.

Each option is a triple — an id and two labels — and the English label is literally the phrase the model receives:

instrument: [
  { id: 'piano',  en: 'warm piano',                    zh: '温暖的钢琴' },
  { id: 'drums',  en: 'punchy live drums',             zh: '有力的实鼓' },
  { id: 'pad',    en: 'lush synth pads',               zh: '宽广合成器铺底' },
  { id: 'vinyl',  en: 'vinyl crackle and tape warmth', zh: '黑胶噪点与磁带暖声' },
  // ...
]

Two reasons this beats a textarea:

1. The phrases are written in the vocabulary these models were trained on. "vinyl crackle and tape warmth" carries production meaning; "sad vibes" carries almost none.
2. The output is stable and translatable. Because the phrase is data rather than a string glued together from user input, the same choice can be rendered in a second language without asking the user to write a prompt in it.

Decision 2: cap the instruments at four — and say why

Our first version allowed unlimited instrument chips. Prompts ballooned into "piano and acoustic guitar and electric guitar and strings and brass and synth pad and drums and bass", and the mixes got muddier rather than richer: every timbre you name competes for the same arrangement space.

So the cap is four, enforced at selection time rather than at submit time:

function onInstrumentToggle(id, checked) {
  const idx = state.instruments.indexOf(id);
  if (checked) {
    if (idx === -1) state.instruments.push(id);
    if (state.instruments.length > MAX_INSTRUMENTS) {  // MAX_INSTRUMENTS = 4
      state.instruments.shift();                       // drop the oldest, not the newest
      showHint('Up to 4 instruments — we dropped the oldest one to keep the prompt focused.');
    }
  } else if (idx !== -1) {
    state.instruments.splice(idx, 1);
  }
}

The UX detail matters more than the number. Silent truncation reads as a bug; a four-second hint that says which chip was dropped and why reads as a rule. Users immediately re-pick instead of filing a complaint.

Decision 3: a prompt is not a translation

Roughly half of our users write in Chinese. The naive approach — build the English prompt, then translate the template — breaks in three places: clause order, punctuation, and connectors.

Same configuration, two builders:

Pop at a slow 75 BPM. Warm piano and a soaring string section. A warm and romantic mood.
Female lead vocals sung in Chinese. Theme: missing someone on a long train ride home.
Written as a personal gift song for someone special, about 120 seconds.

流行风格，慢板 75 BPM。配器：温暖的钢琴、磅礴弦乐。情绪：温暖浪漫。
女声主唱，中文演唱。主题：在回家的长途列车上思念一个人。
作为送给亲友的定制歌曲，时长约 120 秒。

Three things a string-level translation would mangle:

• Order. Chinese groups instruments and mood as labelled fields (配器：, 情绪：) and puts the theme before the intended use. English reads better as genre → instruments → mood → vocal → theme → use.
• Connectors. English needs a, b and c (plus a / an agreement); Chinese uses 、.
• Punctuation. Sentences end with 。, clauses join with ， — not ASCII . and ,.

So the builder assembles clauses per language, and each vocabulary entry carries both labels. One data structure, two grammars, nothing to keep in sync.

Show the budget instead of hiding it

If a limit exists, put it on screen. The builder renders a live counter (184 / 500 characters) and turns it into a warning once the cap is passed.

Free text is the only field that can blow the budget, so the idea input is hard-capped at 120 characters — maxLength on the element and a slice() in the state handler, because a pasted string can exceed what maxLength allows in some browsers:

ideaEl.setAttribute('maxlength', String(MAX_IDEA));
ideaEl.addEventListener('input', () => {
  state.idea = ideaEl.value.slice(0, MAX_IDEA);
  renderPrompt();
});

The rest of the choices are enumerable, so the worst case is computable: four instruments, all phrases at their longest, still lands under the cap. That's the real payoff of a controlled vocabulary — the UI cannot build an invalid prompt.

No backend at all

The builder is one static page with no server round trip. Pressing Use this prompt checks whether you're signed in, then either hands the prompt to the app via a query parameter or stashes it in localStorage and routes through login:

fetch('/api/auth/me')
  .then(r => (r.ok ? r.json() : null))
  .then(data => {
    if (data && data.user) {
      location.href = 'dashboard.html?prompt=' + encodeURIComponent(prompt);
    } else {
      localStorage.setItem('pendingPrompt', prompt);  // survive the signup detour
      location.href = 'login.html';
    }
  });

Keeping the tool zero-cost is what let us ship it as a standalone free page — and it's the page we're happiest to hand to a stranger, because there's no signup wall in front of the output.

What I'd change next

• Prune the vocabulary with data. The option lists are hand-authored guesses. Counting which chips actually get used (aggregate counts only) would let us delete dead options and keep the four-instrument cap honest.
• Structure tags. We don't offer [Verse] / [Chorus] placement yet, and that's the next big lever on vocal output quality.
• One tooltip per dimension. Most people don't realise that naming a tempo band changes the arrangement more than naming an instrument does.

If you want to poke at the builder, it's free and runs entirely in your browser: the prompt builder at Toolara Music. It's the front door of Toolara Music, if you want to hear what the prompts turn into.