The AI Cost of Marionette

When we introduced Marionette 5, one reply summed up the general mood: ‘Everybody who knows what marionette is has stiff knees and creaky backs.’ Fair enough. By the vibes, Marionette lost a long time ago.

In that post we described how we decided whether Marionette was still worth investing in. The usual advice was about Backbone’s age, community size and hiring pools, and all of it assumed a human would write the code. So we asked the agents which ecosystem they’d want to work in. They liked Regions, predictable lifecycles and clear ownership. But that was the agents’ opinion, and opinions are vibes too.

Then Nolan Lawson asked, ‘Did anyone think to ask the agents?’ He had an agent build the same app in twelve stacks and measured what each one cost and shipped. Marionette wasn’t one of them.

So we ran his benchmark task with Codex, Marionette included, and published the results. Last time we asked the agents what they thought. This time we measured what they did.

What we ran.

The same small Pokémon app: listings, search, filters and detail pages. Four independent Codex builds in each of six stacks: Marionette, Vanilla, React Router, SvelteKit, Nuxt and Angular. This is our Codex cohort, not an extension of Nolan’s model results.

A small task gives us something concrete to repeat: the same requirements, four independently generated apps per stack, and a common acceptance suite. It also leaves room for a closer look at how each agent chose to organize the work.

Agents inherit the internet’s preferences along with its examples, good and bad. Popularity may influence what they reach for; this experiment does not measure their training data or prove that explanation. It measures what they built here.

We keep Nolan’s results order: cost, time, tokens, delivered quality and source size. Charts show all four applications, their median and their full range. This DEV edition includes the full data tables below; the original article has interactive charts with sorting and per-build values.

01 / COST

What did the agent use?

Hypothetical API proxy per build

USD proxy. Four independent builds per framework. Median (minimum–maximum); ranges are observations, not confidence intervals.

Calculated under Standard, short-context API assumptions. Not measured cost, subscription spending or an invoice. Actual dollar charges are unavailable.

At the stated token prices, Marionette’s median equivalent is about 31 cents. Vanilla is about 25 cents, React Router 35, SvelteKit and Nuxt 42, and Angular 45. This is a hypothetical token-only comparison, not what our subscription charged. The distinction matters more than the pennies.

What the dollar estimate does and does not mean

The calculation uses $2 per million uncached input tokens, $0.10 cached and $10 output, the previously verified Standard short-context tariff for the reported model. It assumes those rates apply; per-request context lengths, effective service tier, extra tool fees and separate child usage are not established. Actual subscription debits are unavailable. It is not an invoice.

Generation time per build

minutes. Four independent builds per framework. Median (minimum–maximum); ranges are observations, not confidence intervals.

Marionette’s median generation took 5.07 minutes, between Vanilla’s 4.66 and React Router’s 5.55. The full ranges overlap. The timer spans the native agent turn, including its tools and self-tests, but not setup, independent acceptance or Lighthouse. It does not establish a framework’s intrinsic development speed.

Tokens per build

million tokens. Four independent builds per framework. Median (minimum–maximum); ranges are observations, not confidence intervals.

Token totals are final cumulative root-thread input plus output. Cached input is already part of input and is not counted twice. Reasoning detail is not added again. A large cached context and fresh output carry different assumed prices, so token count and the dollar proxy are not interchangeable rankings.

02 / QUALITY

Four working apps.

All four Marionette applications passed all 54 independent acceptance cases. So did the retained peer applications. The comparison starts with working software: listing, searching, filtering and navigating the same Pokémon dataset.

Measurement note: The route and scoring weights were the same for every app. SSR builds could benefit from server warmups, while client-side requests were subject to browser throttling. The results also reflect how each generated app loads its data.

Loading measurements

Home JavaScript transferred

KiB. Four independent builds per framework. Median (minimum–maximum); ranges are observations, not confidence intervals.

Marionette transferred a median 25.6 KiB of home JavaScript, smaller than the framework peers and larger than Vanilla.

Home Total Blocking Time (TBT)

ms. Four independent builds per framework. Median (minimum–maximum); ranges are observations, not confidence intervals.

Home Largest Contentful Paint (LCP)

ms. Four independent builds per framework. Median (minimum–maximum); ranges are observations, not confidence intervals.

Home performance was 96 [92–97]; every route’s automated accessibility median was 100. Those scores are not a complete accessibility audit. Home CLS was 0.111. These charts cover home loading. All six routes are retained in the downloadable data.

All six routes.

Median [minimum–maximum] across four application medians. Each application contributes five measurements per route. The warm-server/cold-browser warning above applies throughout these loading comparisons.

Performance score

LCP (ms)

TBT (ms)

CLS

JavaScript transfer (KiB)

03 / SOURCE SIZE

Maintainability

Formatter-normalized nonblank source lines

lines. Four independent builds per framework. Median (minimum–maximum); ranges are observations, not confidence intervals.

Marionette’s median was 1,012.5 nonblank lines, ranging from 950 to 1,053. It sits close to Nuxt and React Router in this cohort, with Vanilla and SvelteKit below it and Angular above. The chart’s table includes every build.

How source lines were counted

All 24 applications were formatted with Prettier 3.9.10 and Svelte plugin 4.1.1 under one file-selection policy. Counts include comments and exclude blank lines, generated output and supplied fixture files. Original source was not modified.

SvelteKit b2 is 946 lines under this policy. Counting its supplied design-token stylesheet would add 69 lines; that stylesheet is excluded consistently, including its whitespace-only variant. This is a formatted source-size comparison, not a complexity measure.

A little less guessing.

Codex built four working Marionette applications. The median generation took just over five minutes, with a token-only API equivalent of about 31 cents. That gives us a more concrete answer than a framework’s age, popularity, or an agent’s opinion about it.

For this task, Marionette took less median agent time and a lower estimated token cost than each framework peer, while delivering the same tested behavior. Vanilla was leaner still. For a framework that supposedly lost on vibes, that is a useful showing.

Stiff knees, maybe. But the agents didn’t seem to mind. We’re still considering launching Buy Me an Ibuprofen.

Data and method.

Download all verified per-build metrics (JSON) · Download the full six-route assessment and methodology

The downloads below contain the verified per-build assessment. The full evidence package on GitHub includes all 24 applications, their timing and usage receipts, acceptance results, all 720 Lighthouse samples, and the normalized source analysis.

Four applications per framework is a small sample. Each route metric is the median of five mobile Lighthouse samples per application; repeated measurements do not create more independent apps. Codex CLI 0.166.0 reported gpt-6.1-sol, with effort and tier overrides unset, under 4 CPU / 8 GiB container limits.

Collection dates and dependency versions differ. Hosted Marionette documentation changed between the first and second builds, though it stayed stable within each build. No MCP tool invocation was observed, but that does not establish identical exposure. Host activity, model variation and generated architecture limit causal conclusions.

The same route, browser settings and scoring weights apply to every app. Two HTTP warmups create a warm-server/cold-browser measurement: SSR loaders can warm server API caches, while fetching a client-only shell does not run its JavaScript. All SSR apps expose artwork in initial HTML. These are measurements of the generated loading architectures, not isolated framework overhead. The local API hostname mapping also differs in some retained peer measurements, with unquantified impact. Peer values remain unchanged.

Task and results sequence: Nolan Lawson · original benchmark source. The full results and application sources and methodology are available on GitHub.