We distilled a job posting extractor into a 400M model that runs on an 8 GB GPU

I run Job Opportunities API (JOA) on my own: a jobs data API with one rule, every posting comes straight from the employer's own systems. Today that is about 2.69 million live postings, each one confirmed at its source within the last 18 hours.

This week we published the model we built to read those postings: joa-job-extractor-400m-gguf, Apache 2.0, free to use. This post is about why it exists, how we made it, where it is good, and where it is not.

Why we needed it

Most of our cleaning is rules: parsers, mappings, regular expressions. Rules are cheap, fast and auditable, and for most fields they are excellent. Salary is where they break down.

Employers write pay in prose. "From 52,000 rising to 58,000 after probation." "Up to 4,200 euros gross a month", written in German. "Competitive, plus a 15% bonus." Our salary parser is precise when it fires, but in an audit of 886 postings checked by hand against the live pages, it found the pay in only 59% of the postings that actually stated one.

A large language model reads that prose easily. The problem is scale and money. Running a big hosted model over millions of postings, again every time a posting changes, is not something a one person company can afford. And we did not want our data quality to depend on someone else's API being up and cheap.

So the goal was a model small enough to run on hardware we already own (one GTX 1070 with 8 GB), good enough to catch the pay our rules miss, and honest enough that we could check every number it returns.

A strange lesson

Before training anything, we ran a long bake-off: the same 100 postings, many models, scored field by field against reference answers. We expected the biggest, newest models to win. They did not, or not by much.

• Even the reference model did not fully agree with itself: run twice, it matched its own answers 96.1% of the time.
• Fast "flash" models (GLM-5.3 Flash, DeepSeek V4.1 Flash) scored around 0.93, a little ahead of the dense Qwen 3.8 27B at 0.926.
• Giving a small model web tools made it much worse: our fine-tuned 3B went from 0.888 reading the posting to 0.520 when it was allowed to browse.
• And in the end, our 400M student matched the 27B general model exactly: 0.926.

That taught us the idea this whole project rests on: a bigger model is often not the answer. The right input (a clean evidence bundle) and narrow training on one job matter more than size. It was the start of our theory, and the 400M model is the proof.

How we built it

1. Evidence bundles. For each posting we build one text input with everything we know, in a fixed order: the rendered page a candidate sees, the JSON-LD JobPosting if there is one, the record from the employer's applicant tracking system, simple liveness signals, and our own stored text as the lowest priority. Typical bundles are 4,000 to 5,000 tokens.

2. A teacher writes the labels. A larger model, GLM-5.3-Flash (MIT licence), read about 47,300 bundles and answered in one flat JSON schema with 13 fields: is the job still open, title, company, location, salary minimum, maximum, currency and period, posted date, agency or not, remote type and seniority. A grounding filter then removed or corrected values the posting text did not support.

3. A student learns them. We fine-tuned IBM's Granite 3.1 1B-A400M, a mixture of experts model with 1 billion parameters in total and about 400 million active per token. LoRA on the attention projections only, rank 64, one epoch. Then we merged the adapter and converted it to GGUF for llama.cpp, in three sizes: Q4_K_M (0.82 GB), Q8_0 (1.42 GB) and f16 (2.67 GB).

Where the GPUs came from: EuroHPC

None of this would have happened on our own hardware. The teacher runs, the training and the evaluations ran on Discoverer+ in Sofia, a EuroHPC supercomputer with NVIDIA H200 GPUs, through the EuroHPC AI Factories Playground access, which is made for startups and SMEs.

The honest version, because I wish someone had told me: we were accepted within two days, but real access came almost a month later, and the "onboarding" was one email. The clusters are shared and busy, so a job can wait in the queue for a day, and you learn Slurm the hard way (always set --time, and never ask for more memory than you need, because you are billed for what you allocate). Still, I know of no other way for a small European company to get that many H200 hours. If you are building something in Europe that needs real compute, apply.

Results

On our test set of 100 held out postings, scored strictly field by field:

The quantised files lose almost nothing, so the 0.82 GB Q4_K_M is the practical choice.

On the wider audit (1,084 postings from 74 job sources, 886 checked against the source):

It also produced valid JSON for all 1,084 postings, including German, French and Dutch ones. Title, company and city land between 95 and 99.

Speed: about 2 seconds per posting on the GTX 1070, one at a time. On one H200 with 32 requests in parallel it did about 62,900 postings an hour. On a CPU it works but is slow (tens of seconds per posting), so treat that as a batch tool.

Where it is weak

• It sometimes invents a number. About 3% of the salary amounts it returned (8 of 271) did not appear in the posting. Our rule: accept a salary only if its digits can be found in the source text. That one check removes the problem.
• Pay period mix ups. When a posting gives both an annual and an hourly figure, it can attach the wrong period. Check that the period fits the amount.
• Country and seniority are weaker than our rules (F1 91 against 99, and 64 against 88). We do not use it for those.
• It only works well with the exact prompt it was trained on. The system prompt is in the repository; use it unchanged.

How we use it

Not as the main extractor. Rules go first, because they are precise and explainable. The model runs second, only where the rules found no pay, and every value it returns must pass the "digits are in the text" check before we store it. In our API every field also says where it came from: published by the employer, inferred by us, or missing.

Try it

# download the 0.82 GB file and the prompt
huggingface-cli download JobOpportunitiesAPI/joa-job-extractor-400m-gguf \
  joa-job-extractor-400m-Q4_K_M.gguf system_prompt.txt --local-dir .

# serve it with llama.cpp (24K context covers long postings)
llama-server -m joa-job-extractor-400m-Q4_K_M.gguf -c 24576 -ngl 99

Then send the system prompt (with {today} replaced by the fetch date) and one evidence bundle to the OpenAI compatible chat endpoint, at temperature 0. The model card has the bundle layout and a real worked example.

Why we gave it away

Because a model like this is only useful to people who already have the job posting text, and that is exactly the hard part of the problem we sell. If it helps your scraper, your research or your job board, good. If you need the postings themselves, already cleaned, deduplicated and confirmed at their source, that is what Job Opportunities API does, with a free key and an MCP server for AI agents.

Questions, results on your own data, or bugs: [email protected].

Loukas Tzekos, founder, Job Opportunities API