Why mocking your Vector DB in CI is a terrible idea (and a 200ms fix)

Over the last year, building a RAG application has become incredibly easy. But if you’ve tried to move that prototype from a Jupyter Notebook to a production-grade CI/CD pipeline, you’ve probably hit a brick wall.

Testing vector search in a GitHub Actions runner without melting the server is a logistical nightmare. When setting up pytest for an app that relies on ChromaDB or Qdrant, engineering teams usually fall into one of two traps.

Trap 1: The Mocking Illusion

You decide to mock the database client using unittest.mock. Your test asserts that collection.query() was called with the right parameters.

Why it fails: You aren't actually testing anything useful. The entire point of a RAG pipeline is semantic search, the cosine similarity thresholds, embedding dimensions, and chunk retrieval. If you mock the vector database, you are flying blind. You’ll deploy to production only to realize your distance metrics were completely wrong.

Trap 2: The DinD Heavyweight

You decide to do it right and use Testcontainers to spin up a real ChromaDB instance in your CI workflow.

Why it fails: Running Docker-in-Docker (DinD) in a GitHub runner (which gives you 7GB of RAM) is painfully slow. Pulling gigabytes of vector database images and booting the daemon consumes massive amounts of memory. Your CI pipeline goes from 45 seconds to 10 minutes, or simply crashes with an OOMKilled error.

Separating Compute from State

The problem isn’t the database; it’s where we are running it. We are forcing our CI compute runners to also act as infrastructure provisioners.

The ideal architecture for testing is separating the two. Your CI runner should do nothing but execute your test code. The database should live on a remote, isolated server, boot instantly, and destroy itself the moment the test finishes.

To solve this, I built TrashDB. It’s an orchestration API (built in .NET 10 running on bare-metal) designed to do one thing: spin up databases in 200 milliseconds and kill them.

I recently shipped a native Python SDK so you can integrate it directly into your pytest fixtures without writing complex bash teardown scripts.

import pytest
import chromadb
from trashdb import TrashDBClient

@pytest.fixture(scope="session")
def vector_db():
    client = TrashDBClient()

    # Spins up an ephemeral ChromaDB container in ~200ms
    container = client.create_container(engine="chromadb", ttl_minutes=5)

    yield container

    # Cleans up the remote server automatically
    client.terminate_container(container.id)

def test_rag_retrieval(vector_db):
    chroma_client = chromadb.HttpClient(
        host=vector_db.host, 
        port=vector_db.port
    )

    # Your actual semantic search tests go here...

By externalizing the database, your runner only executes Python code. No heavy image pulls, no OOM crashes, and you can run multiple jobs in parallel without data collisions.

I’m building TrashDB in public and it's currently in a free Alpha. If you are struggling with testing your AI apps in CI, grab an API key at trashdb.dev.

Are you currently team mock, team Testcontainers, or something else entirely? Let me know below.