Projection maps of my website's RAG vectors, and what they could not tell me
I wanted to see what the vectors behind my website's search actually looked like. This is a companion to a longer article on my site, which has the maps, the exact-neighbor comparison, the chunk diagnostics and a figure
I wanted to see what the vectors behind my website's search actually looked like. This is a companion to a longer article on my site, which has the maps, the exact-neighbor comparison, the chunk diagnostics and a figure pack with the method notes. Here is the method and the two things I keep coming back to.
What I plotted
On 6 November 2026 I took a snapshot of the site's published index: 85 pages, split into 1,128 text chunks, each a 512-dimensional embedding from openai/text-embedding-3-small. I normalized each vector to unit length, kept the metadata aligned with its row and checked that all 85 sitemap pages were present. There were no empty chunks, no zero or non-finite vectors and no exact duplicate chunk bodies. The plots were calculated from the existing vectors; no new embeddings were generated.
I drew the index in two dimensions three ways. PCA gives a linear baseline, t-SNE emphasizes local neighborhoods, and UMAP builds its view from a neighborhood graph. The t-SNE example uses cosine distance, perplexity 30 and seed 42; UMAP uses cosine distance, 30 neighbors, minimum distance 0.1 and the same seed.
First, measure the map
Before reading anything into a cluster, I checked how much of the original neighborhood each map kept. For every point, I compared its ten closest neighbors on the map with its ten closest vectors in the original space.
| Projection | Ten nearest neighbors retained |
|---|---|
| PCA | 13.1% |
| t-SNE | 75.1% |
| UMAP | 64.5% |
These percentages measure the map's fidelity, not search accuracy. They describe these settings on this corpus, not a general ranking of the methods. What they told me is that the islands are places to investigate, and that their spacing and apparent size are not reliable semantic measurements. An isolated point might be a distinctive passage, an index page or an extraction problem, and only its text can tell those apart.
Then read the text behind the dots
I confirm a map's suggestion with exact cosine neighbors in all 512 dimensions, and then I read the chunks.
The strongest cross-page pair in the index had a cosine similarity of 0.944. Reading it showed repeated article listings on the Writing index and the MCP series page. The similarity was understandable; it was not evidence of two independently written passages reaching the same conclusion. That is why I read the text before treating a group as a topic or a pair as unwanted duplication.
The chunk diagnostics explain much of the map's shape. 81.4% of the ten nearest neighbors come from the same page: longer articles contribute more points, passages from one page often share a subject, and the embedding input includes the page title and heading path. The snapshot also has five chunks below 200 characters and 24 at the 1,500-character cap. Inspecting adjacent chunks on one page found a URL split after https://docs.strapi.io/cms/fea, with tures/admin-tokens) beginning the next chunk. That is a concrete boundary defect. Its effect on retrieval still needs testing.
What this is not
This is an inspection of a website retrieval index, not an evaluation of a complete RAG answering system. The site combines vector and keyword results, so the exact neighbors are vector similarities, not the site's search results. Before claiming retrieval quality, I would test representative questions against reviewed answer-bearing passages and compare the actual ranked results. The visual checks tell me where to look and what to test next.
The full article, with the figures and the method notes: https://pournasserian.com/writing/visually-validating-rag-vectors
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.