Recasting a 15-second meme with five faces: why the last two seconds needed their own pass

The "Outstanding" desk-concert meme blew up on TikTok in October: a soul band squeezed behind an office desk, a singer in a headset going all in, a red keyboard, two backup singers and a sax solo to close. People wanted it with their own friends in it, so we built a small tool for that: upload up to five photos, get the same 15-second clip back with those faces in the five seats.

This post is about the parts that were not obvious: building a clean base clip, getting five faces into the right seats, and one short shot the video model refused to handle.

1. Build the base clip once, not per order

The viral original features real, recognisable people and team logos. We did not want to ship either, so the first step was a one-time "recast": generate a version of the clip with fictional people and plain clothes, keeping the exact shots, cuts, timing and audio.

That base clip is what every order starts from. Two mistakes we made along the way:

• The group shot had too many people. We passed one group photo as the reference for "everyone in the back", and the model happily put all five people from that photo into a shot that only has three. Fix: one single-person reference image per seat, and instructions that state the exact head count per shot ("the group shot has exactly three people; do not add anyone").
• A logo survived in the first second. The opening shot is a side profile and the model kept the original patch on the hoodie. A second pass over the base clip with "remove every logo, badge and text from clothing" fixed it.

Because the base clip is generated once, those mistakes cost a few test runs, not one per customer.

2. Five seats, described by shot

The clip has three camera shots:

The first instructions described each seat by appearance ("the man in the blue shirt gets photo 3"). It mostly worked, except that features started leaking between seats: the keyboard player got one photo's beard and another photo's glasses and shirt.

What helped was describing seats by where they appear, not only by what they look like:

keyboard seat = the person sitting in front at the red keyboard in the group shot,
who is also visible at the edge of the final sax shot → photo 2
...
each photo is a different person; never combine the face, hair, glasses
or clothes of two photos into one person

After that change, four of the five seats landed correctly on the first try.

3. The 2-second shot the model kept ignoring

The fifth seat, the sax player, did not work. Three different instructions, three runs, and the sax close-up at the end always came back with the stand-in's face. Worse, the sax photo's features showed up on the keyboard player at the edge of that shot.

Our reading: a 2.2-second shot at the very end of a 15-second clip simply carries too little weight in a single generation pass.

So we stopped asking one pass to do everything:

1. The main pass fills lead, keyboard and both backup singers, with the sax seat left out.
2. A segment pass runs on a 3-second cut of the base clip (12.0–15.0 s) with only the sax photo (plus the keyboard photo, because he is visible at the edge). On a short clip where the sax player fills the frame, it works on the first try, and it costs a fraction of the main pass.
3. A splicer puts them together: main video up to the cut at 12.833 s, then the segment from its own 0.833 s mark (the same cut point) to the end, with the main video's audio on top.

The segment starts 0.8 s before the cut on purpose: the model gets a little lead-in context, and we throw that part away.

4. The splicer: 60 lines of Python around ffmpeg

The web app runs on a serverless platform with no ffmpeg, so the splice runs on a small server we already had. The flow:

• When both jobs finish, the order goes into a splicing state and is added to a short queue.
• A cron job on the server runs every minute: it lists queued orders over an authenticated endpoint, downloads both videos, splices them and uploads the result. The order then flips to done.
• If the server is down, the web app delivers the main video (without the sax swap) after 10 minutes, so no order gets stuck.

The core of the splice:

norm = f'scale={W}:{H}:force_original_aspect_ratio=increase,crop={W}:{H},setsar=1,fps={fps:.4f}'
parts = [
    f'[0:v]trim=start=0:end={at},setpts=PTS-STARTPTS,{norm}[p0]',
    f'[1:v]trim=start={skip}:duration={D - at},setpts=PTS-STARTPTS,{norm}[s0]',
    f'[p0][s0]concat=n=2:v=1:a=0,tpad=stop_mode=clone:stop_duration=0.2,trim=duration={D}[v]',
]
subprocess.check_call(['ffmpeg', '-y', '-i', main, '-i', seg,
    '-filter_complex', ';'.join(parts),
    '-map', '[v]', '-map', '0:a?',            # audio always from the main video
    '-c:v', 'libx264', '-crf', '20', '-pix_fmt', 'yuv420p',
    '-c:a', 'aac', '-map_metadata', '-1',      # drop container metadata
    '-movflags', '+faststart', '-t', f'{D}', out])

A few details that mattered:

• Normalise both inputs (size, SAR, fps) before concat, even if they "should" match. Generated clips can differ by a pixel or a fraction of a frame rate.
• tpad + trim=duration guarantees the video is exactly as long as the audio. Without it, the segment's slightly different length left a black frame or cut the last beat of the sax.
• Take audio from one source only. The original song runs across the cut; mixing two generated audio tracks would make the cut audible.
• -movflags +faststart so the result starts playing on a phone before it is fully downloaded.

On the server the splice takes about 14 seconds for a 15-second 480p clip. The whole order, from payment to download, came in at about six and a half minutes.

5. Results and what we would do again

• One base clip, generated once, reused for every order. It is the single most important decision: a clean, logo-free base makes every order cleaner.
• Describe seats by shot and screen position, not just appearance, and say explicitly that images must not be mixed.
• When one short shot keeps failing, cut it out and give it its own small job instead of fighting the instructions.
• Keep the fallback boring: if the splice step is unavailable, deliver the main video after a timeout.

If you want to see the result, the tool is live at outstandingaivideo.net: the example on the page was made from five AI-generated test portraits, one per seat, with nothing edited afterwards.