Nanbeige4.3-33B Benchmark Attempt: Mac Speed and Memory

Every Nanbeige4.3-33B benchmark score we could find traces back to the lab's own model card. So we set out to run the 33B dense model ourselves, on a Mac Studio with MLX, through our Gauntlet sb-7.2 and Forge 2.0 agentic coding benchmarks. In both, an agent has to build or upgrade a real app. It never got past our pre-run checks. Our MLX serving engine, the server the model runs in, threw away its tool calls, and the largest prompt our 96 GB Studio served was under a quarter of the length the Gauntlet's conversation reaches. I can give you a measured speed curve on an M3 Ultra, a fit table for Macs from 16 GB to 128 GB, and the command to serve it correctly today. By our own calculation, 128 GB is the smallest Mac that fits our Gauntlet, on a Mac running nothing else. The second model, Particle-1, came out level with one DeepSeek V4.1 Flash snapshot and behind the other on our Gauntlet, and it cost more than both. On two Forge 2.0 builds each, Particle-1 averaged above both snapshots, though 1017 kept the best single build.

The tool-call failure was a bug in our engine; when we asked the model directly, it wrote correct calls. The memory wall is partly the model's design. It's a dense 33B model with a big KV (key-value) cache, the memory that holds everything the model has already read. The rest is our memory guard, the check that refuses a prompt it thinks won't fit, which counted the engine's own disk-cache memory as somebody else's and then refused prompts it could have served. I'll call that the self-count problem, because the guard was in effect counting the engine against itself.

Particle-1 is Particle AI's coding model on OpenRouter, post-trained on DeepSeek V4.1 Flash and sold on token efficiency. On one Gauntlet run each it scored 0.5873, against 0.5910 for DeepSeek's September snapshot and 0.6957 for the later 17 October snapshot that DeepSeek calls 1017. It earned less than the September run before our scorer capped both at the same ceiling, set by one failed check. It also cost more than both, because only about half its prompt was billed as cache hits. Its two Forge 2.0 builds scored 0.653 and 0.5717, against 0.2893 and 0.379 for September and 0.2422 and 0.6618 for 1017.

Nanbeige4.3-33B benchmark: why it has no score yet, and what did run

Our Gauntlet sb-7.2 asks an agent to build a payments console with two Python services, vendor webhooks and a 3D view in 150 model calls; Forge 2.0 asks it to upgrade a Jira Forge app in 300 calls, tested on emulated Jira sites. Both run inside our agent harness, the desktop coding app we build, and local models run through our MLX engine, a fork of the open-source Rapid-MLX server. MLX is Apple's machine-learning framework for its own chips.

Before a local model gets a 5 to 15 hour benchmark slot, we run a readiness probe: is it the right build, do tool calls work end to end, and what's the largest context the Mac really holds? For Nanbeige that took about 1 hour 20 minutes on our test Mac Studio (M3 Ultra, 96 GB). It passed the build check and failed the other two.

Ranks are from the live benchmark board, read again on 6 November; the board is the current word. Each DeepSeek Forge 2.0 cell holds the scores of two builds.

Mostly our own bugs kept Nanbeige4.3-33B from starting

Its tool calls vanished, and the parser was ours

Our first check sends a request shaped the way our harness sends it (its system prompt, two tools, "create hello.py then run it") and looks for a tool call in the reply. Nanbeige's came back broken in the non-streamed response. In the streamed one, the path our harness uses, it was missing entirely. The non-streamed content looked like this:

<tool_call>
function=developer__text_editor>
<parameter=command>...

See the missing < before function=? No parser recognises that. So we rendered the official template ourselves and asked the model directly through /v1/completions. Decoding was greedy, so it always took the most likely next token. It wrote clean qwen3_coder XML:

<tool_call>
<function=developer__text_editor>
<parameter=command>
write
</parameter>

So the model's output was right, and the problem was on our side. Our engine builds a tool grammar, a filter that only lets the model write tokens that fit the expected call format. It builds it from the parser it auto-picks, and the log says which one: Tool-grammar warmup complete (LLTokenizer pre-built for parser 'hermes'). Hermes expects JSON after <tool_call>, so the grammar masked out the model's <, and the sampler then picked a padded vocabulary id the mask didn't cover. (Nanbeige's config declares 166,912 ids, more than the tokenizer has.) The log again: token id 166911 out of range; stopping, then dropping grammar constraint (free-form fallback). Everything after that decoded freely, minus the bracket.

Why hermes? Our engine matches the model's path against a table of name patterns, and the first match wins; its nanbeige row maps to hermes and was written for Nanbeige4.1-3B. Nanbeige4.3's card says to serve it with --tool-call-parser qwen3_coder, and that row was never updated when the format changed. A table that matches on names has no way to notice a change like that, so I think the template should win over the name, as it already does in mlx_lm, the standard MLX library and server.

Stock mlx_lm looks inside the chat template, sees <tool_call> followed by <function=, and picks qwen3_coder. To prove the cause, we symlinked the same model folder under a name containing "qwen3.6", which hits an earlier row in our table. The engine reported tool_call_parser qwen3_coder_xml. Then the model passed a 3-turn round trip (write the file, run it, summarise) streamed, non-streamed and with thinking on.

We kept the renamed folder off the board. I'm glad we did. An entry served under a fake model name needs explaining on every run page. The fix belongs in the engine anyway. We've queued two changes: a nanbeige4.3 row ahead of the old one, and a grammar mask that covers the padded ids.

An open thread on the model page (#34) blames stray newlines in the chat template, but I don't think that's what bit us, since the official template gave clean XML. Thinking will be off when Nanbeige runs. Our engine does that on tool requests that don't ask for it, and our harness doesn't ask. The card evaluates "at their highest thinking effort", so I'll put that gap next to any score we publish.

212,992 bytes per token, and a Gauntlet that needs 322,851 tokens

Nanbeige4.3-33B is a plain dense transformer. Its config lists LlamaForCausalLM, 52 layers, 56 attention heads and 8 KV heads, each 128 numbers wide. Each token in the context keeps a key and a value per layer and per KV head, and in bf16 that's 2 × 52 × 8 × 128 × 2 bytes, or 212,992 bytes per token. Grouped-query attention, where several attention heads share one stored key and value, already makes that 7 times smaller than 56 full heads would, and it's still more than 200 KB a token.

Our cloud runs on this Gauntlet peaked at a 322,851-token prompt, so that's the window a local model needs to get the same room. At 212,992 bytes a token that's 68.8 GB of KV cache. Add 35.2 GB of 8-bit weights and a couple of gigabytes of workspace, and our guard arrives at roughly 106 GB. macOS lets the GPU use only part of the RAM by default. Our Mac Studio's GPU memory limit had been raised to 86 GB before this, which lets the engine use about 86 to 87.5 GB when idle, and a stock 96 GB Mac gets about 81 GB, so both fall well short of the roughly 106 GB the guard asks for.

It wasn't idle. Other sessions on the Mac held 23 to 36 GB during the probe, and the engine's ceiling is physical memory minus "others" minus a 5 GB floor. That put it at 68 GB at mount. At the first refusal it was 53 GB, because 19.6 GB of the engine's own disk-cache memory had landed in "others" on top of the other sessions:

This prompt (128085 tokens) needs 79.7 GB but this Mac can hold 53.3 GB; the maximum context here is 128084 tokens.

DeepSeek V4.1 Flash quotes 890 bytes of global KV per token on its card, thanks to compressed sparse attention that stores its cache in FP4 (a 4-bit number format). Nanbeige's figure is about 239 times that. The two mechanisms differ, so it's only a rough comparison, but it shows why one model takes a million tokens easily and the other struggles at 128k.

int8 KV was our first choice, and it was the wrong one

8-bit KV looks like the fix, so the probe started there. On paper it brings the 322,851-token cache down to about 36.5 GB (113,152 bytes a token, including the scale numbers int8 needs to restore its values). On our engine it went the other way.

Our memory guard kept pricing the KV at the bf16 rate even with int8 live. Our app's memory planner, a separate check, priced int8 correctly, but the engine's guard ignored the planner's number. So int8 saved live memory and bought no admitted context: its window, 128,619 to 175,872 tokens during the session, was smaller than the 227,328 that bf16 got later.

Prefill, the reading of the prompt before the first token comes out, got slower too. int8 needed extra scratch memory while reading the prompt and bf16 didn't. To stay under the ceiling the guard kept halving the prefill chunks, 17 times by 20:33, and it measured that scratch space at 224 bytes per chunk-and-context unit with int8 and none with bf16. That's 56 heads times 4 bytes, so my reading is that the int8 path builds full fp32 attention score rows where bf16 uses fused attention.

Then we turned int8 KV off. The same steps ran on bf16 with app settings otherwise unchanged, and bf16 decoded 18% faster at 10k, 37% faster at 31k and 60% faster at 64k, and peaked lower. On this engine int8 is worse for Nanbeige on every axis we measured, so I'd run it on bf16, and both our Nanbeige profiles are back there. A fix needs a guard that prices the KV type the engine actually uses, and an int8 path with fused attention.

Our memory guard counted the engine against itself

With bf16 the guard's own estimate for a 128,123-token prompt was 64.7 GB, and the quiet Mac had a 73.7 GB ceiling at 21:11, so by my reading it should have fitted. It was refused twice, each time after about 8.5 minutes of prefill: "needs 64.8 GB but this Mac can hold 56.0 GB", then "needs 64.7 GB ... can hold 50.4 GB". Both refusals came from our guard's arithmetic, not from the Mac running out of memory.

Our engine's status endpoint shows why. At the end of the 64k turn the engine moved 13.1 GB of cached prompt to its disk cache. While it read that back, the memory the engine used outside the GPU (18.84 GB, then 17.18 GB) was filed under "others", next to the browser and the other sessions on the Mac. That's the self-count problem: the guard shrank its ceiling by the memory the engine was using itself, then refused the prompt for not fitting. It held for about ten minutes the first time, and it came back during the retry's prefill.

An earlier engine release meant to take the engine's own process out of "others", and this disk-cache memory slipped through. There's a related bug. The RAM budget for cached prefixes, the stored start of the conversation that lets the next turn skip re-reading it, was 0.00 GB all session, so every turn's prefix went to disk and back. Both are queued with the parser fix. Swap rose from 1.4 GB to 3.6 GB and nothing crashed, but the bug still cost us the 128k measurement twice.

Running Nanbeige4.3-33B on a Mac Studio with MLX: speed, memory and fit

We used fcmeyer/Nanbeige4.3-33B-MLX-8bit. It was converted from the official repo at revision c1093c5, and upstream hasn't changed the weights, config or tokenizer since. It's uniform 8-bit affine with group size 64, 35,189,757,302 bytes, and the converter reports 98.76% top-1 agreement with bf16 on 563 tokens, meaning it picked the same next token as the 16-bit original that often. Nanbeige hasn't published an MLX build. (Why MLX and not GGUF? GGUF is the model file format llama.cpp uses, and we compared the two formats on the same model in MLX vs GGUF on Apple Silicon.)

Speed comes from two growing sessions on our Mac Studio, one per KV setting, run the same way; each request carried our harness's system prompt and two tools, and each turn added a document. Then it asked the model to find a fact we had hidden in the text. It found it every time. Engine settings were those of our app, with the prefix cache on and thinking off.

Our cold 75,925-token request waited 895 seconds, almost 15 minutes, before the first token came back. That last row is int8, but it's the one to plan with, because bf16 never got a cold request that large. In a growing session the wait is much shorter. Only the new part gets read. The 64k turn waited 270 seconds for its 33k new tokens on bf16, and 434 on int8.

On a dense model, decode is mostly limited by how fast the Mac can stream the weights. At the M3 Ultra's 819 GB/s of memory bandwidth, reading 35.2 GB of weights per token caps it at about 23.3 tokens a second before any KV reads, and we measured 17.9 at 10k, about three quarters of that. Past 64k we only have projections. Fitted to the bf16 rows, they give about 9.5 tokens a second at 128k and 5.4 at 322,851. A cold 322,851-token prefill would take about 1.7 hours, longer than DeepSeek V4.1 Flash 1017 took for its whole Gauntlet run.

Serving it today with mlx_lm

If you want Nanbeige as a local agent now, I'd use stock mlx_lm.server. It picks the tool parser from the template:

mlx_lm.server --model fcmeyer/Nanbeige4.3-33B-MLX-8bit --port 8080

We read that template check in mlx_lm 0.31.3, and it's still there in 0.32.2. To check the parser, send one tool request. A reply with a tool_calls field is fine. Raw <tool_call> text in the content, or a call that begins function=, means the parser isn't wired. Size your agent's context from the fit table below. At bf16 KV the 524k-token maximum context on the card needs more memory than any Mac in that table has.

What we didn't test: mlx_lm.server itself. Our probe never went through it, so I can't give you its speed there, and LM Studio and the GGUF builds under llama.cpp are unchecked too.

Which Mac fits Nanbeige4.3-33B

This table is computed. It comes from our own measurements, and only its 96 GB row was checked against a real Mac, further down. It uses our measured 8-bit weights (the 4-bit and 6-bit sizes are the published file sizes of third-party builds), the measured KV cost per token and our guard's 2 GB overhead. It assumes the GPU can use 81% of physical memory on 48 to 96 GB Macs and 84% on 128 GB. For 32 GB and below I guessed 67%. Our guard then keeps 97% of that. Apple counts memory in binary gigabytes, so every row converts it to the decimal gigabytes the table uses (96 GB is 103 GB, 128 GB is 137 GB). The conversion is why the 96 GB row comes out at 81.0. In each cell the first number is bf16 KV, which our engine admits today whatever you set; the second is int8 if the engine priced it correctly and used fused attention, which is unproven. 524k is the model's native limit, so no cell goes higher.

We checked the 96 GB row against our Mac. It shows what a busy machine costs: 205k on paper at 8-bit, 64k to 76k served. The difference is the other sessions and the self-count problem, so I'd read the table as the best case on a Mac running nothing else. For our Gauntlet at 8-bit, a 128 GB Mac is the smallest that fits. The 4-bit build on 96 GB falls short at 283k. If I were buying a Mac to run this model as an agent, I'd start at the 128 GB row and I wouldn't trust the rows below it without measuring.

What a full run would cost in time

We projected this from the speed curve and the token counts of earlier runs on the same benchmarks. One Gauntlet comes to about 12 to 18 hours: decoding 254k to 363k tokens at around 7.4 tokens a second, plus prefill and about 150 prefix loads from disk. Two Forge 2.0 builds come to about 8 to 10 hours. Thinking on makes both longer. It's slow, but I'll still run it on a 128 GB Mac with bf16 KV once the parser and self-count fixes ship.

What the card claims, and what we can't check yet

Nanbeige's card calls it "the flagship teacher model for the Nanbeige4.3 series" and compares it with Qwen3.8-27B, Muse-Glimmer-30B and Gemma4-31B, "evaluated at their highest thinking effort". In Nanbeige's table it leads Qwen3.8-27B on Terminal-Bench 2.1 (80.9 against 73.0, with Terminus 2) and DeepSWE (65.5 against 42.5), where SWE means software engineering. It trails Qwen on SWE-Bench Pro (60.3 against 61.7) and on GDPval (84.8 against 86.4). GPQA is a graduate-level science quiz. On its Diamond set Nanbeige edges Qwen, 89.5 against 89.2, and trails Gemma4-31B at 91.3. These are the lab's numbers, on its harness. One user in a "Don't waste your time with this model" thread put most of the bad feedback down to "misconfiguration and having insufficient memory for context". The second half fits what I saw.

Licence and where you can get each model

The fcmeyer MLX build does ship an Apache 2.0 LICENSE file and credits "Nanbeige Team, Nanbeige4.3-33B (2026), Apache-2.0". If your legal team wants the text from the source, I think they'll have to ask the lab.

Particle-1 vs DeepSeek V4.1 Flash: which snapshot it was trained on, and both on our board

Particle-1 launched on 28 October from Particle AI, the inference company Suhail Doshi founded, and OpenRouter describes it as "a token-efficient agentic coding model from Particle AI, post-trained on DeepSeek V4.1 Flash". It's API only. Particle trained "a rank-64 LoRA adapter covering the experts in all layers", a small add-on of trained weights worth about 1.25% of the parameters. Its reward "incentivizes the model to solve tasks successfully while using fewer tokens for reasoning and tool calls".

Particle's launch post doesn't name the DeepSeek snapshot. The base scores it quotes (GPQA Diamond 90.91, BabyVision with tools 89.60) match the September V4.1 Flash card, while the 1017 card gives BabyVision 92.5. So my guess is September, but that's inference. Both snapshots are on our board: the September one from 4 October, and 1017 from 18 October, written up in our DeepSeek V4.1 Flash 1017 benchmark. DeepSeek calls 1017 "a minor update" with better coding and agentic scores, so the tables below show both.

Particle's own chart, read from its data

The post's headline is "around ~45% fewer tokens on average across a wide variety of tasks". The chart data in the page has four coding benchmarks. All four ran with the Terminus 2 agent, 16 attempts (rollouts) per task, a 512k-token limit and max reasoning.

Particle-1 passes fewer tasks than the base on three of the four and more on one. The post says pass rates "are within the confidence intervals of the base model", and the confidence-interval field in the data is empty. A rough interval from the counts in the data puts all four gaps, 1.4 to 3.5 points, inside the noise. We did that arithmetic ourselves, and it ignores that the 16 rollouts per task are correlated, which makes the real intervals wider. So I think the claim is fair. Its four token reductions average 40.5%; the post doesn't say which task set gives ~45%. Its data also labels the DeepSWE panel "partially_reported" and its token and accuracy fields "user-supplied-simulated", and we couldn't find an explanation on the page.

Its cost numbers check out too. Recomputed from the same data, cost per task falls 34.4% to 51.6%, exactly the post's range, at the fixed 95% cache hit rate the post assumes. Our own Particle-1 run was billed at about half that cache rate. Particle's post also reports a contract-review test that puts the trade in numbers: at 8x adapter strength, with the add-on's effect turned up eight times, "output fell by 78.7%, but only three of eight reports passed the strict check". So shorter output can cost passes. I wanted to see what that does on a task Particle never trained for.

Particle-1 on our Gauntlet: level with September, behind 1017

Our Gauntlet scores in two layers. A model earns a score from its 99 checks, and a critical failure multiplies that down. Then the worst kind of failure it has sets a ceiling on top, and capping squeezes everything above the ceiling into a 5% band, so runs under the same ceiling are close to a tie whatever they earned.

Against the September run, Particle-1 finished only 0.004 behind, and both hit the ceiling set by the check that the payment field is visible on screen. Before the ceiling, Particle-1 earned 0.7679 and the September run 0.8403. If all 51 runs are ordered by that pre-ceiling score, Particle-1 is 26th and the September run 19th, against 29th and 27th on the board. Most of that 0.07 gap is the approval multiplier: on the checks alone it's 0.8145 against 0.8403, a 0.026 gap. Either way it's inside the spread we see between two builds of one model, so with one run each I call them level. Against 1017 the gap is 0.108, and I'd put most of it on the ceiling, 0.599 against 0.699.

The setups weren't identical. Our app asks for medium reasoning effort. Particle's docs map that to High for Particle-1. DeepSeek lists no medium level, so 1017's real effort was the provider's choice, and the September run went out before our app asked for medium at all. Particle-1 ran on our app's release 3.0.149, and 1017 on 3.0.118, 29 releases earlier, while the scorer version was the same for all three.

One more thing about that 0.9429 multiplier: the message is our scorer's, and it overstates the failure. The scorer says "dead primary flow — approval cannot complete through the UI", but its own probe shows the approval went through (state "approved", the payment appeared); what was missing was the notification in the UI. It cost 0.0024 of the final score, because the ceiling was already binding. I think the penalty is harsher than the bug, and I'd rather say so here than let the table imply Particle-1 broke the approval flow.

One draw call without its index list set the ceiling

Halfway through, Particle-1 found the browser in our test kit slow, because it draws 3D in software (SwiftShader). At 22:37:10 it wrote "The instanced path is pathologically slow in this browser. Rewriting the draw path to a single merged geometry buffer with a colour texture". Its towers are built from quads. Each has four corners and six indices (web/viz.js:272-276):

function pushQuad(b, p0, p1, p2, p3, factor, band) {
  ...
  b.idx.push(base, base + 1, base + 2, base, base + 2, base + 3);

The merged path copies the 76 corners per tower and then draws them as a plain triangle list, never touching the index list (web/viz.js:821, and the same at 913 for the pick pass):

gl.drawArrays(gl.TRIANGLES, 0, state.count * VCOUNT);

I read that line as the bug. Our scorer matched 31 of 120 predicted pixels for the seeded payments, where it needs 95%, and clicks on six towers picked the wrong payment six times. Meanwhile the tower count (12,291) and three of its four geometry sums matched the contract, because the data was fine and only the drawing was wrong. My reading, from the code: 76 corners don't divide into triangles that way, so each face loses about half its area, and from the second tower on, triangles straddle neighbouring towers. Our scorer's own screenshot below shows the smeared field; we didn't re-render a corrected build to prove the cause.

1017 made the same switch at its tool call 78, and its version drew correctly. It wrote "The renderer is bottlenecked by SwiftShader's instancing. I'll switch the field to a baked non-instanced vertex buffer with a per-tower colour texture", and its field passed the visible-field check and every pick and height check. I can't show 1017's draw code, though. We deleted its build tree after the October run, so everything about 1017 here comes from its run page, its surviving logs and our notes from 18 October. We kept no tree or request logs for the September run either, so its real reasoning effort is unknown.

We re-graded Particle-1's run with the scorer's own functions. With the visible-field check passing, three failures of the next-worst kind still bind, and the score comes to 0.6273, which is 0.07 behind 1017 on one run each, inside the spread we see between two builds of the same model.

Its own probe caught the bug, and it called it occlusion

Five minutes before it finished, Particle-1's own camera probe printed this at 22:50:15:

"pick": {"want": "pay_00005", "got": {"id": "pay_00022", "index": 22}}

Its thinking at 22:50:41: "that's occlusion (a tower in front). Fine — the grader compares against the analytically nearest surface, which accounts for occlusion. My pick is GPU truth ✓." Our scorer disagreed. Its pick check is exactly that comparison, and it scored 0 of 6; our scorer's own clicks also picked other nearby towers (pay_06698 came back as pay_06722), so it looks like the same failure. That probe output was the cheapest place to catch the draw-call bug, and Particle-1 explained it away in one line of thinking. Its closing summary at 22:55:07 still listed "GPU picks agreeing with projection" among its results. At 22:53:40, looking at its final screenshot, it wrote: "The field is nice but a bit "carpet-like". Acceptable."

Where Particle-1 beat 1017: the close-up tower and the coast

In the same burst of thinking, Particle-1 also caught a real subtlety in the spec. Dragging right lowers the camera's yaw, its left-right angle, so the coast (the glide after you let go) has to keep lowering it: "vyaw must be negative for a rightward drag". It changed const vyaw = 0.30 * prev.dx / prev.dt; to const vyaw = -0.30 * prev.dx / prev.dt; and checked the decay against its own samples. The coast check scored 1.0. Its inspection spire used the index list. It passed the tower-geometry and framing checks, and its backend checks in the first two tiers all scored 1.0, including 384 of 384 Berlin-day buckets across the daylight-saving switch.

1017 lost exactly those. Its coast scored 0.8, its octagon loop drew 6 of 8 wedges (the check that set its ceiling), and 0 of 8 inspector poses framed the tower properly. I read the two runs as mirror images: Particle-1 got the close-up right and broke the overview, and 1017 got the overview right and clipped the close-up.

Forge 2.0: two builds, ahead of both snapshots on average

Particle-1's first Forge 2.0 build made 7 calls, and then OpenRouter answered every request with a 429, "temporarily rate-limited upstream", with the reason fleet_capacity_exhausted, for 32 minutes, so we cancelled it. We relaunched early on 3 November. That gave us two builds: 0.653, 5th of the 72 runs on the board, and 0.5717, 11th. 1017's better build scored 0.6618, so I'd still give it the best single result of the three, by 0.009. Particle-1's two builds were 0.08 apart. 1017's were 0.42 apart.

Forge 2.0's final score is the tests earned, times a multiplier for critical defects, times a reliability multiplier. For reliability, the scorer takes each group of tests and multiplies the score by 1 − 0.10 × the shortfall of that group's worst test, so a group whose worst test fails completely costs 10%. Neither Particle-1 build had a critical defect. Both earned more from the tests than three of the four DeepSeek builds did. Then reliability cut every build. It took 32% and 38% off Particle-1's two, and more than that off three of the four DeepSeek builds.

I count 1017's $0.320 as a lower bound, because our records mark that bill incomplete. Particle-1's cache rate was better here than on the Gauntlet, at 79% and 90%, and still under every DeepSeek build. As with the Gauntlet run, I can't tell from our side whether that's the host or our app. Its two builds were billed $1.26 together, against $2.39 for September's and at least $0.61 for 1017's. Its builds wrote 165k and 199k output tokens, against 178k to 225k for DeepSeek's. Two builds each is a small sample. I'd read this as Particle-1 being steadier than 1017 on this task, and higher on average than both snapshots, which is a weaker claim than being the better model.

Particle-1 pricing, and what each run cost us

Particle-1 has three published prices. I couldn't find a page that explains the difference.

On Particle's own API, Particle-1 costs the same per token as the DeepSeek V4.1 Flash 1017 it also hosts, so there the whole saving is in the tokens. On OpenRouter, Particle-1 lists below DeepSeek's own host for 1017, which our 1017 run was pinned to ($0.15 input, $0.60 output, $0.003 cache reads, read on 3 November). Particle AI also serves 1017 on OpenRouter at exactly Particle-1's price, so on paper Particle-1 should have been the cheaper run. It cost 2.78 times as much as 1017.

I'd put it down to the cache. Only half of Particle-1's prompt was billed as cache hits. That left 9.48 million tokens billed as fresh input, about 79% of the bill by our arithmetic on the listed rates. Priced the same way at a 99.7% hit rate, Particle-1's tokens would have cost about $0.155, less than 1017 (a hypothetical, and our arithmetic).

An earlier launch on the same app version also cached only 49%. Our harness stopped that one at call 42, after about 8 cents, because one OpenRouter billing lookup had come back 404 and its spending check treated that as a failure. We didn't publish that run; the score above is from the relaunch. Both DeepSeek runs were on older app versions, so it's either how this host caches our traffic or something in how app 3.0.149 sends it, and from our side I can't tell which.

The token story depends on which snapshot you hold it against. Against 1017, Particle-1 used 20 fewer calls and about 10% less output, while per call it wrote slightly more (1,507 output tokens, 787 of them reasoning, against about 1,448 and 730). Against the September run it wrote 29% less output and 37% less reasoning over the whole run. Per call that's 24% and 33% less (1,507 and 787 tokens against about 1,990 and 1,173), which is closer to what Particle measured, though that run probably thought at DeepSeek's default effort. On the Gauntlet, with one run each, Particle-1 didn't beat either DeepSeek snapshot on score or on the bill. Forge 2.0 gave it the better average score. Its two builds there cost less than September's and about twice what 1017's were billed. I wouldn't switch to it on the price list alone.

Where we were wrong: we probed a week late

Most of what stopped Nanbeige was ours, and we found it late. Its weights had sat on our SSD since 26 October. The parser row, the int8 pricing and the self-count problem would all have shown up in the same 80-minute probe a week earlier. I should have run it the day they arrived. On the Particle side, I still can't explain the cache rate, and it's the number that decides whether Particle-1 is cheap for you.

What I'd check before trusting either model

For Nanbeige on a Mac, I'd check which tool parser your server picked before blaming the model. On our engine and Rapid-MLX, /v1/models shows tool_call_parser; on vLLM or SGLang it's the --tool-call-parser flag you launched with; mlx_lm.server picks it from the template. The card's answer is qwen3_coder. Then do the KV maths for the context your agent really uses, and measure int8 against bf16 on your own engine before assuming 8-bit is a win.

For Particle-1, I'd measure the cache hit rate on your own traffic, and test it against the DeepSeek snapshot you'd otherwise use, on your own task. Particle's chart says fewer tokens for slightly fewer passes. On our Gauntlet it came out level with one DeepSeek snapshot and behind the other, and on Forge 2.0 its two builds averaged above both.

If you've run Nanbeige4.3-33B locally on any engine, how much context did you really get, and did tool calls work out of the box? And if you've put a token-efficient model like Particle-1 behind your own agent, did the shorter answers cost you retries later? The comments are open, and so is our Discord. Nanbeige's row goes up on the board once the engine fixes ship and it gets its 128 GB Mac.

Originally published on leanzero.net. More Atlassian, Forge and local-AI write-ups at leanzero.net/blog, and if you're planning a migration or a Forge app, that's what we do: leanzero.net/services.