I mapped 290 documented failures in AI document tooling (LangChain, RAGFlow, Docling...) - feedback welcome

Hi guys,

I’ve spent the last couple of years building a legal AI platform. We’ve acquired several clients, including established law firms working on complex arbitration cases. And given the kind of documents these firms deal with, we’ve had to process some pretty challenging material.

Along the way, we invested a significant amount of time hardening our stack: reliable document ingestion, retrieval, source-linked citations, long-running agents, and handling all sorts of unexpected user behaviours.

What struck me is how much effort went into making the foundations reliable, before we could even focus on what actually differentiated our product.

I tend to think of this as two layers: the foundation layer (ingestion, retrieval, citations, agent runtimes, etc.) and the value layer (the features and workflows that make your product useful and different).

Ideally, most of our effort should have gone into the value layer. Instead, we spent months stabilising the foundations.

I recently did some research and found that many builders are running into similar problems with existing tools and frameworks (LangChain, LangGraph, RAGFlow, Docling, etc.). I compiled 290 documented incidents from public discussions, grouped by problem, with links to the original sources.

You can explore them here: https://rachis.io/pain-map

This got me thinking: what if we made parts of our production-hardened stack available to other builders, so they could spend less time fixing infrastructure and more time building what actually differentiates their products?

We’ve already open-sourced our local AI stack, and we’re exploring which other components would be useful to expose.

I’m curious whether this resonates with anyone building document-heavy AI applications.

If you’ve run into similar issues, I’d love to hear what cost you the most time. And if you’d be interested in using some of these components, feel free to DM me or leave your email at https://rachis.io. I’ll reach out when we have something to try.