AI fatigue is real. So is the tiny offline model doing my most private task

My feed has a rhythm now. Every morning: a new model, a new agent framework, a new benchmark chart with a bar that goes up and to the right. By lunch, three hot takes. By evening, a thread about how the benchmark was wrong. I understand why "AI fatigue" is one of the most-reacted topics on dev.to this week. I feel it too.

So this is not a post about a new model. It is a post about the opposite experience: picking the smallest model that would run offline, giving it one boring job, and finding out that this is where the technology quietly works.

The boring job

I deal with Swiss administrative letters: payment orders from the debt-collection office, tax decisions, health-insurance adjustments. They are written in a formal register that even native speakers have to read twice. I wanted a tool that explains them in plain Italian.

The obvious path was an API. The obvious problem was the document itself. A Zahlungsbefehl contains your full name, address, the creditor, and exactly how much you owe. It is not something you upload to a server you know nothing about. So the real requirement was: it has to run entirely on my machine.

The smallest thing that works

The whole tool is about 306 lines of Python 3.10, standard library only. No pip install, no SDK, no API key, no account. It talks to Ollama over HTTP on localhost, where gemma3:1b does the language work. One billion parameters, on CPU. No GPU, no queue, no per-token cost.

This is the part that surprised me, after months of reading that everything interesting needs a frontier model: for a bounded job, the small model is not a compromise. It is the correct size.

What the model does, and what it never touches

The split is the whole design.

The code - plain, inspectable Python - handles everything that must be exact. It parses the letter, pulls out the amounts, the deadlines, the reference numbers, the office that sent it. Deterministic extraction, the kind of thing a regular expression and a date parser have done reliably for decades.

The model never sees a number it could invent. It receives the extracted fields and the letter text, and it does the one thing it is genuinely good at: explaining, in calm everyday Italian, what this letter is and what the sender is asking for. The deadline in the output is the deadline the code extracted. The model can paraphrase a sentence; it cannot paraphrase a date into existence.

That boundary is what makes the output trustworthy enough to actually use.

The one number I'll give you

On my machine, on CPU, one measured run: about 40 seconds to explain one payment-order letter end to end. That is exactly one measurement on exactly one machine - your hardware will differ, and I will not pretend otherwise.

Is 40 seconds slow? For a chatbot, yes. For a letter that sat in a drawer for a week because I did not want to read it, it is nothing. The right question was never "is it fast". It was "is it fast enough for the job", and the answer is comfortably yes.

What this taught me about the fatigue

The fatigue, I think, comes from the framing that AI is one thing that should be everywhere, doing everything, all the time. That framing produces demos that impress and products that exhaust.

My experience was the inverse. One model, one billion parameters, one machine, one job with clear edges. No account, no upload, no subscription, nothing to cancel. The tool is boring, and boring is exactly what a letter from the debt-collection office deserves.

If you are tired of the discourse, this is my suggestion: pick one small, private, slightly annoying task of yours. Put the smallest local model on it. Keep the numbers in the code and the words in the model. It is a very calm way to find out what this technology is actually for.

The tool described is real and personal. The 40-second figure is a single measurement on a single machine, stated as such.