Talking-Head Video From One Still and One Audio Track: Two Digital-Human Routes Compared
A talking-head video normally needs a camera, a person, a quiet room and a few retakes. This episode does it from a single still image plus an audio track, and gets to 2K without hand-animating a frame. Four ComfyUI wor
A talking-head video normally needs a camera, a person, a quiet room and a few retakes. This episode does it from a single still image plus an audio track, and gets to 2K without hand-animating a frame.
Four ComfyUI workflows make up the chain, and all four run on cloud ComfyUI, so nothing heavy needs to be installed locally.
The four workflows
| # | Workflow | What it does |
|---|---|---|
| 1 | Voice clone + TTS (CosyVoice3) | Clone a voice from a short reference clip, then generate narration from text |
| 2 | Digital human, OmniRef 8-step | Image + audio drive mouth, head motion and eye line; distilled sampler keeps it fast |
| 3 | Digital human, Pika 6-step | The faster six-step route for the same task |
| 4 | Restoration + 2K upscale (SeedVR2) | Restore and upscale the finished clip to 2K |
Why two digital-human routes instead of one
OmniRef and Pika are not interchangeable settings — they are two different chains that happen to solve the same problem. Running both on the same image and audio is the point of the episode, because their failure modes differ:
- OmniRef drifts on head rotation. Turn the head far enough and the jaw line stops matching the source portrait.
- Pika drifts on the eye line. Direction of gaze wanders before rotation does.
Picking the better route from a spec sheet is guesswork. Running both on your own material takes ten minutes and tells you which failure you can live with.
The pipeline order matters
- Voice first. Clone from a short reference, generate narration text. The audio is the driver for everything downstream, so it decides timing.
- Then the digital human. One portrait plus that audio. No keyframes, no masks, no separate lip-sync pass.
- Then the finish. SeedVR2 restoration and upscaling so the talking head is not the softest thing in the edit.
Change the narration later and you regenerate the head, because the mouth is driven by audio timing. That is the one structural limitation worth planning around.
Files
- HF: https://huggingface.co/AI-KSK/minimax-h3-digital-human
- ModelScope: https://modelscope.cn/models/Sunjian520/minimax-h3-digital-human
Video walkthrough (Chinese): https://youtu.be/2cgAuD5c_vI
Bilibili: https://www.bilibili.com/video/BV1qTWJ6kEw3/
Notes
- Tutorial release, not a product. Node parameters are the ones used on screen.
- The narration in the language editions is AI-assisted translation plus AI voice synthesis, stated in the video description too.
- Not affiliated with or endorsed by MiniMax, Alibaba, ByteDance or RunningHub.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.