Miktam Bible | Part 2 — An AI Study Assistant That Runs on the Phone

Most AI features in apps are a thin wrapper around a cloud API. Miktam Bible's study assistant is different: the language model runs on the phone, and every answer is grounded in verses pulled from the offline database.

🧠 The model

The assistant runs SmolLM2-135M, a small model quantized to 4 bits in GGUF format (about a 105 MB one-time download), through llama.rn, the React Native binding for llama.cpp. The settings are chosen for phones:

context = await initLlama({
  model: nativePath,
  n_ctx: 2048,        // Mobile-safe context window
  n_batch: 256,       // Mobile-safe batch size
  n_threads: 4,       // CPU threads for inference
  n_gpu_layers: 0,    // 0 = CPU only
  use_mlock: false,
  use_mmap: true,     // Memory-map the model file
});

CPU-only with a memory-mapped model file keeps it working across the wide range of Android hardware. If the model has not been downloaded yet, or fails to load, the chat shows a banner and switches to a limited offline mode with pre-written study answers. Download the model for real, verse-grounded answers.

📚 Answers grounded in Scripture (RAG)

A small model on its own will happily make things up. So every question goes through a retrieval step first:

Your question
  └── 1. Search the Bible (FTS5, with synonym expansion and stop words removed)
        └── 2. Take the 10 most relevant verses
              └── 3. Build the prompt: those verses + your question, in the template the model was fine-tuned on
                    └── 4. Generate the answer on the phone
                          └── 5. Turn verse references in the answer into tappable links

The model sees the retrieved verses as context for your question, and any reference in its answer (for example John 3:16) becomes a link back into the reader.

🔮 What's next: hybrid search

Keyword search finds verses that share words with your question, not verses that share its meaning. The pipeline is already shaped for hybrid search: FTS5 results plus vector similarity (sqlite-vec), merged with reciprocal rank fusion. The missing piece is a small on-device embedding model (MiniLM); until it lands, the assistant uses FTS5 alone.

🔒 Private by design

No API keys and no server: your questions are never sent anywhere. Once the model is downloaded, you can put the phone in flight mode and the assistant still works.

Next, Part 3: reading plans and reminders that keep up with you.

📥 Get Miktam Bible

Miktam Bible is in Google Play closed testing:

1. Join the AI2TH Google Group with the Google account on your phone. Members are testers for every AI2TH Play app.
2. Opt in at play.google.com/apps/testing/miktam.bible.
3. Install from Google Play.

Different Play account? Request access here.

• 🐙 GitHub: github.com/AI2TH/miktam_bible
• 🌐 Website: ai2th.github.io/miktam.html