Ming-3.0-tts on a 16 GB card: the GPU sat mostly idle, and why a bigger one barely helps

When a speech model runs slower than real time, the reflex is to rent a bigger GPU. With Ming-3.0-tts-0.5B that buys very little. An RTX 2000 Ada sat idle more than two thirds of the time. An RTX 4090 worked at most one moment in five, and the speedup was about 17%.

The model comes from the inclusionAI team at Ant Group. They released weights and code under MIT, with their own benchmarks and a frank list of limitations. It copies a voice from a few seconds of sample audio, builds one from a text prompt describing it, and turns inline tags like [Laughter] or [Sigh] into sounds. Their model page only mentions H800-80GB and H20-96G cards, with no memory figure. We ran their unmodified code on a small card to get one. Every number is in our Ming-3.0-tts memory and speed test on an RTX 2000 Ada.

Low GPU utilisation is the first thing to check

"Real time" is a ratio: seconds of audio produced per second of compute. A 20.7-second English paragraph cost 31 to 32 seconds on the RTX 2000 Ada. On the 4090, a 20.9-second paragraph cost 27 seconds. Its faster host processor may explain part of that gain.

The culprit is the decode loop, which builds the audio step by step, 80 milliseconds per step. Between steps, Python code on the CPU prepares and launches the next one. Keeping up with playback takes 12.5 steps per second. We measured about 8.3 to 8.9. The GPU is a fast bus stuck behind a slow ticket check at every stop.

Before sizing hardware for a text-to-speech (TTS) model, run nvidia-smi while it speaks and read utilisation, the share of time the GPU is working. If it stays low, a faster card will not buy you speed. One possible way out, untested by us: vLLM-Omni, an engine built for fast serving, which Ming's roadmap marks as supported.

Emulate the small card before you rent it

No real card below 16 GB ran. Two 12 GB RTX 3080 Ti machines booted without the SSH access our installer needs, and no 8 GB card was available to rent. So we faked one. PyTorch, the library Ming runs on, has a call, torch.cuda.set_per_process_memory_fraction, that limits how much GPU memory the process may take. We lowered it on the 16 GB card until loading broke.

With caps of 8, 4 and 3 GiB, the output WAVs had the same md5 checksum as the uncapped run, and speed did not change. At 2.75 GiB, setup died with "CUDA out of memory". Add roughly 0.4 GB for the driver and the article's estimate lands at a 4 GB card. Nobody ran that on real hardware, and a smaller chip may also be slower.

Copy the hashing habit: loading under a cap proves little, matching bytes prove the cap changed nothing.

Log the timings with every request

Our custom backend, a short Python file that wraps the authors' code behind an HTTP API, returns its own measurements next to the audio. This came back for a cloned line with three tags on the RTX 2000 Ada:

"timings": { "gpu_s": 11.02, "realtime_factor": 0.63, "peak_vram_mib": 2922 }

That request spent 11.02 seconds of GPU time on 7.0 seconds of audio. Strip the tags and the same line took 8.05 seconds for 5.3 seconds of audio. Tags are extra sound, and sound costs time. Log these fields from day one.

Three traps you will hit

• Numbers. Before speaking, English text goes through normalisation, a step that spells numbers and symbols out as words. It read 4521 as "four thousand five hundred twenty, one", pause included, and turned the phone number 555-0142 into a subtraction. Spanish text bypasses that step, and "1.249,99 euros, un 12,5 %" turned into nonsense. Spelling English numbers as words did not help. Check every price by ear.
• Tags by language. The authors document tags for English and Chinese. We only tried them in English, so tags in Spanish text remain untested.
• The OpenAI speech route. OpenAI-style apps request speech from a dedicated endpoint, which on InferenceKey cannot yet carry a custom backend's audio (open, IK-531). The example goes through the chat route and decodes the WAV out of the message text.

What the test does and does not show

We ran each clip through Whisper, OpenAI's speech-to-text model: 17 of 19 lines matched word for word across English, Spanish and Chinese, and both misses were numbers. A speaker-recognition model rated the clones 0.87 in English and 0.84 in Spanish, both cloned from an English reference. Keep in mind that Ming relies on that same model to capture a voice, which tilts the score towards Ming. No human listened, each line ran once, and the reference was a clean synthetic clip from Kyutai's Pocket TTS. The backend handles a single request at once, so concurrent load is untested.

My read: good for recorded narration, wrong for a live voice agent. If you need Spanish speed more than cloning or tags, our Spanish Pocket TTS voice on CPU reached 1.05x real time on a desktop CPU. Nobody has compared the two for quality.

From empty caches, the backend was ready to serve after 60.3 seconds, about half of it installing Python libraries. The article's appendix has the setup, per-request tables and VRAM ladder for Ming-3.0-tts.

Credit for the model goes to inclusionAI at Ant Group, to OpenAI for Whisper, to Kyutai, who made Pocket TTS, and to the AI Search channel, whose news video pointed us to Ming. The SDK example, the audio samples and every measurement are in the Ming-3.0-tts research article on inferencekey.com.