Dev.to AI 🤖 Ai 👁 0 📖 2 min read

Same 50 ms as ripgrep, but it returns answers: benchmarking a local memory index on 557 docs

I keep a local memory index for my AI coding agent. Last week I wrote about why (it kept forgetting the same design decision). This post is the boring part everyone skips: the measurements. I ran the same corpus through

I keep a local memory index for my AI coding agent. Last week I wrote about why (it kept forgetting the same design decision). This post is the boring part everyone skips: the measurements.

I ran the same corpus through two tools on the same machine, three runs each, and took the median. No cloud, no GPU, no vector database.

The setup

  • Corpus: 557 markdown files, 3.1 MB of text pulled from a real project (docs, specs, runbooks)
  • Indexing result: 4,137 chunks, database 11.8 MB, 9.2 s to build on first run
  • Machine: Windows, 16 indexing workers (8 used), nothing else running
  • Query: five ordinary queries -- mcp config, license activation, hybrid search, index checkpoint, rate limit
  • Timing: end-to-end CLI process time (startup + open DB + query + print JSON), median of 3 runs. I did not strip startup, because your agent pays that cost too.

Results

Tool What it returns Median
cortex search (BM25 / FTS5) top-10 ranked chunks with snippets 45-54 ms
ripgrep -l file paths that contain the literal string 43 ms

Same order of magnitude. That surprised me -- I expected the index to lose on a corpus this small.

What the numbers actually mean

  1. Speed is not the differentiator. At ~50 ms both tools are "instant" to a human and both fit inside one agent turn.
  2. The output is the differentiator. rg -l "mcp config" gives you 9 file paths. You still have to open them, split them, and hope the right paragraph is near the top. cortex search gives you ranked chunks, each small enough to paste into a context window.
  3. Context is the real budget. Ten raw files can be 200k tokens. Ten 512-token chunks are ~5k. The measurement that matters is not latency, it is tokens you did not have to spend.
  4. Honest caveats. 557 docs is a small corpus; ripgrep scales better on millions of files than an FTS index does. I disabled the vector layer (vector.enabled: false) so this is pure BM25 -- with embeddings enabled the numbers would be worse, not better. Warm cache, single machine, one run set.

Why I still run it this way

Because the index also answers questions that a grep cannot: "what did we decide about retries", "where is the license check", "which config key controls chunk overlap". Those are semantic-ish queries where the answer is a paragraph, not a file.

The index is one 33 MB Go binary, no Docker, no Postgres, no embedding service. Indexing 557 docs costs 9 seconds and 12 MB on disk. That is the whole pitch: the cheapest thing that turns your own notes into retrievable context.

If you want to reproduce: install in one command and index any folder.

curl -fsSL https://cortex.qataroasis.cc/install.sh | sh
cortex index ~/notes

Windows: irm https://cortex.qataroasis.cc/install.ps1 | iex

There is a 14-day Pro trial (500k docs / 100 GB) with no payment: https://cortex.qataroasis.cc/trial

If you run the same five queries on a bigger corpus and the index falls behind ripgrep, tell me the doc count -- I want that number more than I want stars.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.