Google's OKF cut LLM hallucination 81% on DFIR telemetry — because the format, not the model, was the problem
DFIR tooling still hands its telemetry to LLMs as one big JSON dump. At scale that is a context-exhaustion engine: latency climbs, the model starts hallucinating, and the analyst has to re-verify everything it said. Our
DFIR tooling still hands its telemetry to LLMs as one big JSON dump. At scale that is a context-exhaustion engine: latency climbs, the model starts hallucinating, and the analyst has to re-verify everything it said.
Our new paper asks a narrow question: does the shape of the serialization matter more than the model?
We benchmarked Google's Open Knowledge Format (OKF v0.1) — a graph-native, selectively traversable format — against flat JSON, using 7,680 inference cycles across five open-source model architectures with escalating decoy densities (four difficulty tiers).
Results, relative to JSON:
- Hallucination suppressed by 81.1% at the hardest tier
- Token overhead cut by 33.6%
- Noise-invariant recall sustained at 0.94–0.99 as decoys increased
The useful part is not "OKF is better". It's the mechanism: when the model can traverse to the relevant subgraph instead of ingesting a monolithic dump, context stays small and grounded — so the failure mode that produces hallucination never gets loaded in the first place.
If your incident-response agent ingests JSON telemetry, this is a format decision you can test before you touch the model.
Paper: https://sushantpoudel2028.com.np/publications/okf-dfir
More writing, including the negative results: https://sushantpoudel2028.com.np/writing
security # ai # dfir # llm
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.