Local RAG on a mini PC: EmbeddingGemma 2, PDF Desk, and when to add a vector store

Local RAG doesn't require a GPU tower humming under your desk. On a mini PC you can assemble a working retrieval pipeline from two published pieces: EmbeddingGemma 2, an open 740M-parameter embedding model, and PDF Desk, a small Python/Streamlit app that answers questions about a PDF and points at the page it used. Vector stores like Qdrant or Chroma are the optional next step — not a prerequisite for getting something running today.

Here's the path, step by step, with the expected result at each stage and the places where it usually goes sideways.

What you need before you start

• A mini PC with enough RAM for a 740M-parameter model. Check this first — it's the one thing pip can't fix.
• A Python environment for PDF Desk.
• macOS or Linux. On Windows the command variants live in the repo README, so the commands below won't line up.
• Hugging Face access, to pull the EmbeddingGemma 2 weights.
• The PDF Desk repository from GitHub.
• One test PDF. PDF Desk handles a single active document at a time.
• Optional: Qdrant or Chroma if you plan to index many documents or non-PDF content.

A checklist I'd actually tick off:

• [ ] EmbeddingGemma 2 weights downloaded from Hugging Face
• [ ] PDF Desk source cloned from GitHub
• [ ] Python and dependencies installed
• [ ] One test PDF ready
• [ ] Free RAM and disk space confirmed on the mini PC

Step 1: Pull EmbeddingGemma 2 from Hugging Face

EmbeddingGemma 2 is Google DeepMind's open multimodal embedding model at 740 million parameters, and it maps text, code, images, video and audio into one vector space. That's the part that matters for RAG: your query and your document chunks land in the same space, so comparing them is a straightforward similarity lookup.

Find the model page on Hugging Face and download the weights into a folder you can point at later. Expected result: model files sitting locally on the mini PC, no API key involved. If the download stalls, check connectivity and confirm the repository is reachable — big files also need free disk space and headroom in RAM.

Step 2: Launch PDF Desk and ask your first question

PDF Desk is a Python application with a Streamlit front end. Clone it, install dependencies, run it, and the interface opens in your browser.

git clone <pdf-desk-repo-url>      # exact URL is on the GitHub page
cd pdf-desk
python -m venv .venv
source .venv/bin/activate          # Windows users: follow the README
pip install -r requirements.txt
streamlit run <entry-file>         # file name as listed in the README

Then upload one PDF, type a question, and read the answer. The detail that makes this useful is the page reference: the response comes back tied to a specific page number, so you can verify the claim instead of trusting the prose. When you need a different document, replace the active file and re-upload before asking the next question.

Step 3: Decide whether Qdrant or Chroma earns its place

Neither source post walks through a vector store installation, so I won't invent one — for setup details, the official Qdrant and Chroma documentation is the place to go. What the posts do make clear is the division of labour:

The pipeline shape is identical either way: EmbeddingGemma 2 turns content into vectors, the store persists them, and queries are embedded into the same space so semantic search works. PDF Desk is the shortest path to a demo; a vector database is what you reach for once "one active PDF" becomes the blocker.

Where it breaks, and the quick fix

• Only one active PDF at a time. That's a documented limitation. Swap the file and re-upload; don't expect a folder-wide index.
• No Windows commands in the walkthrough. Open the GitHub README and use the Windows section instead of copy-pasting macOS/Linux lines.
• Hugging Face download fails. Verify the network, confirm you can reach the repository, and free up disk and RAM for a model this size.
• You want everything indexed. That's the vector store's job, not PDF Desk's.

Quick answers

What is EmbeddingGemma 2? An open multimodal embedding model from Google DeepMind with 740M parameters, combining text, code, images, video and audio in one vector space.

Can a mini PC run local RAG? Yes — EmbeddingGemma 2 is open on Hugging Face and PDF Desk runs locally on macOS/Linux, and both are light enough for a mini PC with sufficient RAM.

Qdrant or Chroma — which one? The source posts don't compare them. For plain PDF search, PDF Desk indexes on its own; for a bigger RAG build, pick based on your preference and follow the official setup guides.

How do page references work? PDF Desk links each answer to the relevant page of the active document, which is the core feature described in the original post.

If you want the original write-ups and more local-AI material, this article was originally published on AI Release — https://ai-release.org — and the channel posts are collected at https://t.me/ai_release1.

Takeaway

Start small: one PDF, PDF Desk, page-cited answers — a fully local RAG loop you can validate in an afternoon. Bring in EmbeddingGemma 2 when you want a custom embedding space, and only then reach for Qdrant or Chroma, once the one-document limit is the thing actually slowing you down.