Dev.to WebDev ๐Ÿ›  Dev ๐Ÿ‘ 0 ๐Ÿ“– 2 min read

I Built MLBox: A Browser Workbench That Shows How LLM Tokenizers Really Work

Hi, I'm Mohit Verma, a Class 10 student from Patna, India, learning machine learning by building things. MLBox is my open-source project: a browser workbench for LLM tokenizers, training intuition, and API pricing. Four

Hi, I'm Mohit Verma, a Class 10 student from Patna, India, learning machine learning by building things. MLBox is my open-source project: a browser workbench for LLM tokenizers, training intuition, and API pricing. Four tools, one static page, zero dependencies โ€” every number is measured in your own browser.

Live demo: https://mohittt-vermaa.github.io/mlbox/
Code: https://github.com/mohittt-vermaa/mlbox

What's inside

Tokenizer lens โ€” Paste any text and watch 7 real LLM tokenizers cut it up, token by token: counts, cost per million characters, a language cost matrix, a boundary diff between any two tokenizers, and a shareable link that freezes the whole comparison.

Micro playground โ€” A neural net with hand-written backprop training live on canvas. Watch gradient descent solve XOR.

No-GPU recipes โ€” 12 curated ML projects that genuinely finish on a laptop or a free Colab session, each with its one gotcha spelled out.

LLM price bench โ€” 13 current models from 5 providers, transcribed from each provider's own pricing page with source links, plus a workload calculator and a real-prompt cost estimator.

The number that motivated all of this

Paste "เคจเคฎเคธเฅเคคเฅ‡ เคฆเฅเคจเคฟเคฏเคพ" ("hello world" in Hindi, 13 characters) and the tokenizers disagree: GPT-4o's tokenizer needs 5 tokens, Llama 3.1 needs 9, GPT-3.5/4's cl100k needs 13. Same two words, nearly 3x the bill depending on which tokenizer sits in front. Token choice is a cost multiplier and a context-window multiplier, and almost nobody looks at it.

Verified, not approximate

The tokenizer engine re-implements both BPE families in about 550 lines of dependency-free JavaScript, and the test suite checks it token-for-token against the upstream Rust implementations: 218,877 tokens, exact match on all seven models. There is also an Android app and a Windows desktop build in the releases, and everything runs offline with no API key and no backend โ€” your text never leaves your machine.

Why I built it

I kept reading claims about token costs from blog posts instead of measuring them. MLBox is my way of learning ML properly: build the tool, verify every number, and share it so the next student can measure things themselves.

If it helps you, a star on GitHub means a lot: https://github.com/mohittt-vermaa/mlbox

โ€” Mohit Verma
All my work: https://mohittt-vermaa.github.io

๐Ÿ“ฐ Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes โ€” full credit and traffic to the original publisher.