Same 50 ms as ripgrep, but it returns answers: benchmarking a local memory index on 557 docs
I keep a local memory index for my AI coding agent. Last week I wrote about why (it kept forgetting the same design decision). This post is the boring part everyone skips: the measurements. I ran the same corpus through
I keep a local memory index for my AI coding agent. Last week I wrote about why (it kept forgetting the same design decision). This post is the boring part everyone skips: the measurements.
I ran the same corpus through two tools on the same machine, three runs each, and took the median. No cloud, no GPU, no vector database.
The setup
- Corpus: 557 markdown files, 3.1 MB of text pulled from a real project (docs, specs, runbooks)
- Indexing result: 4,137 chunks, database 11.8 MB, 9.2 s to build on first run
- Machine: Windows, 16 indexing workers (8 used), nothing else running
- Query: five ordinary queries --
mcp config,license activation,hybrid search,index checkpoint,rate limit - Timing: end-to-end CLI process time (startup + open DB + query + print JSON), median of 3 runs. I did not strip startup, because your agent pays that cost too.
Results
| Tool | What it returns | Median |
|---|---|---|
| cortex search (BM25 / FTS5) | top-10 ranked chunks with snippets | 45-54 ms |
ripgrep -l
|
file paths that contain the literal string | 43 ms |
Same order of magnitude. That surprised me -- I expected the index to lose on a corpus this small.
What the numbers actually mean
- Speed is not the differentiator. At ~50 ms both tools are "instant" to a human and both fit inside one agent turn.
-
The output is the differentiator.
rg -l "mcp config"gives you 9 file paths. You still have to open them, split them, and hope the right paragraph is near the top.cortex searchgives you ranked chunks, each small enough to paste into a context window. - Context is the real budget. Ten raw files can be 200k tokens. Ten 512-token chunks are ~5k. The measurement that matters is not latency, it is tokens you did not have to spend.
-
Honest caveats. 557 docs is a small corpus; ripgrep scales better on millions of files than an FTS index does. I disabled the vector layer (
vector.enabled: false) so this is pure BM25 -- with embeddings enabled the numbers would be worse, not better. Warm cache, single machine, one run set.
Why I still run it this way
Because the index also answers questions that a grep cannot: "what did we decide about retries", "where is the license check", "which config key controls chunk overlap". Those are semantic-ish queries where the answer is a paragraph, not a file.
The index is one 33 MB Go binary, no Docker, no Postgres, no embedding service. Indexing 557 docs costs 9 seconds and 12 MB on disk. That is the whole pitch: the cheapest thing that turns your own notes into retrievable context.
If you want to reproduce: install in one command and index any folder.
curl -fsSL https://cortex.qataroasis.cc/install.sh | sh
cortex index ~/notes
Windows: irm https://cortex.qataroasis.cc/install.ps1 | iex
There is a 14-day Pro trial (500k docs / 100 GB) with no payment: https://cortex.qataroasis.cc/trial
If you run the same five queries on a bigger corpus and the index falls behind ripgrep, tell me the doc count -- I want that number more than I want stars.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.