Can You Run Inkling 2 Small Locally? 17B Active, but Plan on 200 GB

You can't run Inkling 2 Small locally yet. Thinking Machines shipped the 594B model on November 5, 2026 with closed weights on Tinker, and the Apache 2.0 weights are only promised for next week. When they land, my math says a 2-bit GGUF will want 190 to 200 GB of RAM plus VRAM, so one 128 GB box is out. DeepSeek's 552B model, which you can download today, also beats it on eight of the nine coding benchmarks the two share in the lab's model card.

I haven't run it, because the weights aren't public, and Thinking Machines hasn't published local hardware requirements. Every number marked as an estimate below is my own arithmetic.

Inkling 2 Small specs: 594B total, 17B active

Per the announcement and model card, this is a sparse mixture-of-experts model that routes each token to 8 of 384 experts. It takes text and images, outputs text, "does not support audio", and supports MXFP8 and NVFP4 numerics.

The 17B figure tempts people into comparing this to a 17B model on a gaming GPU. That comparison breaks immediately, because 17B only describes the compute for each token. The router picks different experts as it goes, so all 594B have to be somewhere you can read them quickly. A July guide to the smaller Inkling put the offload penalty this way: "every token still reads expert weights over the PCIe bus."

Context is murkier, with three numbers floating around. The main RL run used "256k context length" with compaction, the model card says up to 1M, and Tinker's pricing page sells 64K under the base ID and 256K as a separate model ID.

How much RAM and VRAM does Inkling 2 Small need?

Unsloth published memory figures for the two earlier models. Dividing by parameter count gives similar GB-per-billion ratios for both, within about 4% (using the top of Inkling-Small's 4-bit range), so I multiplied those ratios by 594. The last column is my estimate. No Inkling 2 Small quant exists yet, and a new quant recipe could move these figures.

Unsloth also lists a 1-bit UD-IQ1_S build of the 975B model at 270 GB of disk and about 290 GB of RAM plus VRAM. Scaled the same way that's about 177 GB of memory, still too big for one 128 GB box, and I wouldn't start there.

For serving the original checkpoints, the Inkling-Small card asks for at least 600 GB of aggregated VRAM for BF16 and 180 GB for NVFP4. Scaled up, that's roughly 1,290 GB and 390 GB. Thinking Machines hasn't published either number for this model.

Those are floors for loading the model, and context costs extra. DataCamp's Inkling tutorial put the 317 GB Q2 build of the bigger model on four RTX PRO 6000s and used about 85% of GPU memory at a context of just 8,192 tokens. It generated 37.2 tokens/s.

Which hardware can run it: DGX Spark, RTX PRO 6000 or a server

Here's what those estimates mean for actual machines:

On the Hacker News thread, colingauvin suggested "a Q2-Q3 with 2x DGX Sparks. So...$12k at today's prices." I'd only sign up for the Q2 half of that, and even that nervously. My 3-bit estimate is about 19 GB over the 256 GB NVIDIA advertises for two linked units. NVIDIA's own chart rates that pair at "up to 400B parameters" and only reaches 700B at four Sparks. The chart doesn't name a quant, so I read it as a warning more than a hard limit, but 594B is well past it. The closest proof I found is Matt Suiche's September write-up, which got Inkling-Small's NVFP4 checkpoint running across two Sparks on vLLM, with hotfixes for attention kernels and load-time memory. That's a model less than half this size on a different stack, so treat a 594B GGUF on two Sparks as unproven. colingauvin also pointed to people running DS 4.1 Flash on "some GPU + bulk in system memory," guessed "$3k-$4k" for that kind of build and admitted "speeds come down quite a bit."

Software is the other unknown. Inkling GGUFs needed llama.cpp PR #25731, which is still an open draft, and neither Unsloth's page nor the model card says whether 2 Small works with it.

Check your own totals now, so you know which row you're in:

free -g | awk '/^Mem:/{print $2 " GB RAM"}'
nvidia-smi --query-gpu=memory.total --format=csv,noheader
# macOS
sysctl -n hw.memsize | awk '{print $1/1e9 " GB unified"}'

When will the Inkling 2 Small weights land on Hugging Face?

The exact wording: "We intend to make the full weights available under an Apache 2.0 license next week after the completion of external safety testing and red-teaming." Read literally from November 5, that's the week of November 9.

Nothing public backs that up yet. The model card lists the license as "Tinker Terms of Use," and Artificial Analysis has it as proprietary with weights not publicly available. There's no Inkling 2 repo on Thinking Machines' Hugging Face account. The announcement's safety section says "The release of Inkling's weights is pending further safety evaluations by us and our partners," and the lab's open-weights framework says its stages "are not a fixed sequence that automatically ends with releasing the model's weights."

The announcement also discloses that during the main RL run, the lab's monitor flagged the model working around a network block by reaching a third-party service through its search tools. The team paused the run and hardened the infrastructure before resuming.

The precedent is decent. By July 16, Inkling-Small's weights were promised with no date, and the model shipped on July 30. So I expect these weights to appear, and I'd give "next week" somewhat better than even odds. I wouldn't buy hardware against it. Poll for the repo instead:

curl -s "https://huggingface.co/api/models?author=thinkingmachines&search=Inkling" | jq -r '.[].id'

When I checked, that API listed four repos, all from July: Inkling, Inkling-NVFP4, Inkling-Small and Inkling-Small-NVFP4. Watch for a fifth.

Inkling 2 Small vs DeepSeek-V4.1-Flash-1017: DeepSeek wins the coding rows

These come from the lab's own table. It credits outside reporters, including Artificial Analysis for FrontierCode, Datacurve for DeepSWE, Scale AI for SWE-Pro v2 and Vals for Vibe Code Bench, but the lab chose what to show. I only cross-checked the AA Index against AA's site.

The 86.3% on SWEBench Pro v2 Hard is second in its column behind Kimi K3, and DeepSeek isn't in that column at all. On the other four coding rows DeepSeek wins, with a slightly smaller model that Epoch lists as open-weights and that's on Hugging Face now. The model card's longer coding list says much the same: of nine rows with scores for both, DeepSeek takes eight. The exception is Terminal Bench 2.1, 85.1% to 78.4%, a row the card labels as Inkling 2 Small on the Pi harness. HN's markasoftware summed it up as "Not beating dsv4.1 at all." If you're shopping for one local model at this size for coding, DeepSeek is the one to beat.

Outside coding it's mixed. DeepSeek also leads on AA-Briefcase and GDPVal-AA, but further down the page Inkling 2 Small leads it on GDP.pdf (25.2% to 17.2%), DesignArena Website (1,340 to 1,303 in the November 5 leaderboard chart, though the main table has DeepSeek at 1297) and, narrowly, Vals Finance Index (61.5% to 60.9%).

The Intelligence Index score is inconsistent: the announcement's prose says 31, its table says 30.6 under "v5.0," and Artificial Analysis shows 31 under v5.0.1. Probably rounding, but the page doesn't say so, and in that table 30.6 sits below Gemini 3.8 Flash at 30.8.

The lab lists its own weaknesses, which I respect. It says the model "is not the most succinct communicator" and "doesn't always choose the best tools for the job," especially in codebases with many skills. It's also "less optimized for the chat experience," and the lab says it trails closed models on alignment. The main benchmark table leaves out the first-generation Inkling entirely, so you can't read a coding comparison off it. The alignment table does include it, and that's where the gain is clearest: internal misbehavior of 14.9, against 66.8 for Inkling, with lower being better.

What to run this week

Try it on Tinker Playground, or use model ID thinkingmachines/Inkling-2-Small. It's $0.30 per million prefill tokens ($0.03 cached) and $1.00 per million sampled, which undercuts Inkling-Small's $0.58 and $1.44 in the training table. The serverless beta gap is narrower: Inkling-Small is $0.30 and $1.20 there. If you fine-tune now, the announcement says checkpoints "will be downloadable with the release of Inkling 2 Small's weights."

Hosted isn't automatically the cheap option against DeepSeek. Artificial Analysis puts cost per Index task at $1.18, against $0.60 for DeepSeek V4.1 Flash 1017 (Max), which matches HN's Bolwin calling it "about double that of deepseek." Verbosity doesn't explain the gap: AA counts 154M output tokens for Inkling 2 Small and 174M for DeepSeek, at list prices of $1.00 and $1.20 per million. My guess is caching, where AA shows a 90% discount for Inkling 2 Small and 98% for DeepSeek.

If you want a local Inkling tonight, Inkling-Small is your realistic target. Unsloth's command needs that draft llama.cpp build:

./build/bin/llama-cli -hf unsloth/Inkling-Small-GGUF:UD-Q3_K_XL --temp 1.0 --top-p 1.0 --min-p 0.0

Swap the tag for UD-Q2_K_XL if you're on 128 GB. That file is 87.9 GB.

What's your box, and would you bother with a 2-bit build of 594B when DeepSeek's weights are already downloadable?