Dev.to Article #2 Draft（2026-11-09）

Title: The boring math behind a 150-tool AI suite: how I pick models by quality, latency, cost and capability

Tags: ai, machinelearning, productengineering, sideproject

Links: https://arui.ai (brand) · https://arui.ai/ai-image-generator (deep link)

Word count: ~660

The flashy part of Arui.AI is the canvas — 150+ generators for video, image, music and voice sharing one workspace. The unglamorous part, the one that took longer than the UI, was deciding which model sits behind each tool tile.

After months of model selection for a creative suite, I've settled on a four-axis evaluation that beats any leaderboard. If you're building on top of generative models, this framework might save you the spreadsheet hell I went through.

The four axes

Quality — the obvious one, and the one people overweight. Model cards and Twitter demos are cherry-picked. Evaluate on your users' prompts: portraits, product shots, anime, thumbnails. A model that wins abstract art benchmarks can lose badly on passport-photo realism.

Latency — not "how fast is the API" but how fast it feels inside a product. Under ~10 seconds, users wait. Over 30, they tab away. Video generation is the extreme case: anything beyond a couple of minutes has to become an async job with a notification, or the UX is dead on arrival.

Cost — per-generation price, but never in isolation (more on this below).

Capability — the underrated axis. Does the model accept reference images? Can it edit an existing generation instead of rerolling from scratch? Does it hold a consistent character across scenes? A "weaker" model with editing support often beats a stronger text-to-only model in real workflows.

Three tradeoffs I keep making

1. Fast tier vs quality tier. The fastest open image models and the sharpest ones sit roughly an order of magnitude apart in per-image cost. The answer isn't picking one — it's exposing both and letting the task decide. Ideation thumbnails don't need the expensive model; the hero image does. This is why tiered model options beat a single "best" model in creative tools.

2. When the pricier model is actually cheaper. A model that costs more per call but supports instruction-based editing can win on total spend: users tweak one result instead of rerolling twenty. I've watched "expensive but editable" cut total generations per finished asset dramatically. Per-call price is not cost per finished work.

3. Async is a product feature. For video, queue depth and completion time matter more than raw model quality past a threshold. A slightly softer model that finishes in 5 minutes beats a stunning one at 20 minutes for anyone actually producing content on a schedule.

What I stopped doing

• Chasing leaderboards. They measure prompts nobody ships with.
• Evaluating models without a reference-image test. Editing and consistency break ties more often than raw fidelity.
• Optimizing cost per call before looking at cost per finished asset. Rerolls are where budgets actually die.

None of this is rocket science — it's just that each axis pulls in a different direction, and the product's job is to make the tradeoff visible to the user instead of pretending one model wins everywhere.

If you want to see the end result of these tradeoffs, the AI image generator on Arui.AI exposes tiered picks with exactly this logic — fast iterations on one end, high-resolution finishes on the other.

What's your tiebreaker when two models are close? I'm still refining mine.