This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Recap Buddy is a study companion for the oldest learning trick there is: teach it to someone.

You pick a topic, explain it out loud the way you'd explain it to a friend, and Buddy listens. Your browser transcribes you live. When you're done, an open-weight Gemma 4 model reviews only your concepts (never your grammar, accent or "umm"s) and sorts everything you said into four buckets:

Plus a score out of 100 and one thing to revise next. Upload your own notes (PDF, DOCX or TXT) and Buddy grades you against your notes instead of generic textbook knowledge.

At the end of the day, one click writes a first-person Learning Diary page for everything you taught: a score table, what you nailed, what you mixed up, and your top three priorities for tomorrow. Download it as Markdown.

The problem

The Feynman technique works — if you can't explain it simply, you don't understand it. But it has a catch every student hits: there's nobody to explain it to. And even when a friend listens, they can't tell you which parts were correct, which were wrong, and what you completely skipped. That last bucket — the stuff you don't know you don't know — is the one that costs marks in exams.

Who would pay for it

This is the Side Hustle angle, and it's a real one where we live (Pune, India):

• Students preparing for board exams, JEE/NEET, university semesters and interviews — the "explain it back" step is exactly what a ₹500/hour tutor does. Recap Buddy does it every evening for the price of a few cents of inference.
• Coaching institutes and tuition centres — they'd pay a monthly seat fee to give every student a "teach-back" tool graded against the institute's own notes, plus a daily diary teachers can glance at.
• Self-learners and interview prep — explain "how TCP handshake works" or "what a B-tree is" out loud and see exactly what you skipped.

A rough model: free tier with a few reviews a day, a ₹99/month student plan, and per-seat pricing for coaching centres. It works as a business because the model is open-weight and cheap — more on that below.

Demo

🔗 Live app: recapbuddy.vercel.app
🏆 MLH project page: Recap Buddy on MLH

Here's a real review. I deliberately said "plants take in oxygen and release carbon dioxide" and skipped the Calvin cycle entirely — Gemma caught both:

The whole UI is themed like a school notebook: graph paper, ballpoint ink, green ticks, highlighter for "almost there", red pen for wrong answers, sticky notes for what you missed. Buddy the mascot nods along while you talk, its mouth moves with your real mic level, and it reacts to your score. There's also a "Read it to me" button that speaks the summary aloud.

Code

Tejas-Santosh-Nalawade / Hacktoberfest-Hack-Day-ACES

Recap Buddy

Teach it out loud. Buddy checks it.

An open-source study companion that listens while you explain a topic, then uses an open-weight AI model to show what you got right, what was close, what was wrong and what you missed. At the end of the day it writes up everything you learned as a diary page.

Live demo · Demo video · Agent Skill · Report a bug

Contents

• Why Recap Buddy
• Features
• Screenshots
• How it works
• Open-source AI
• Getting started
• Configuration
• API reference
• Project structure
• Deploying to DigitalOcean
• Agent Skill
• Privacy
• Browser support
• Roadmap
• Contributing
• Acknowledgements
• License

Why Recap Buddy

Many students learn best by teaching a concept out loud (the Feynman technique). Others revise four or five topics a day and struggle to recap them by evening. Both run into the same problem:

• There's nobody to explain the topic to.
• Even when someone listens, they…

View on GitHub

MIT licensed. Clone it, run two commands, and you have your own study buddy:

# backend
cd backend && python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt && cp .env.example .env
uvicorn main:app --reload --port 8000

# frontend
cd frontend && npm install && npm run dev   # http://localhost:5173

How I Built It

Browser (React + Vite)                 FastAPI                     Open-weight models
───────────────────────               ─────────────               ──────────────────────────
Web Speech API (en-IN) ──transcript──▶ POST /api/review ──────────▶ Gemma 4 31B (primary)
PDF/DOCX/TXT notes ──────────────────▶ POST /api/extract            │ error / bad JSON?
localStorage (today's sessions)        POST /api/report ──────────▶ gpt-oss-120b (fallback)

Frontend: React + Vite, Tailwind, Motion for animation, react-markdown for the diary. Speech-to-text is the browser's Web Speech API in continuous mode with interim results, and it auto-restarts after silences until you press Stop. Typing is always available as a fallback. No accounts and no database: today's sessions live in localStorage under recapbuddy:YYYY-MM-DD.

Backend: FastAPI + Pydantic, with pypdf and python-docx for note extraction. Every provider speaks the OpenAI chat-completions API, so one openai client covers them all:

PROVIDERS = {
    "digitalocean": {"base_url": "https://inference.do-ai.run/v1/",
                     "model": "gemma-4-31B-it", "fallback_model": "openai-gpt-oss-120b"},
    "gemini":       {"base_url": "https://generativelanguage.googleapis.com/v1beta/openai/",
                     "model": "gemma-4-26b-a4b-it", "fallback_model": "gemma-4-31b-it"},
    "lmstudio":     {"model": "google/gemma-4-e4b"},   # fully offline
    "ollama":       {"model": "gemma4"},               # fully offline
}

Switching from cloud to a laptop with no internet is one env var: LLM_PROVIDER=ollama.

Making an LLM a fair grader

The hard part wasn't calling the model. It was making it grade like a fair teacher. The system prompt sets a few non-negotiable rules:

• Judge concepts only. Input is speech-to-text, so ignore grammar, filler words and transcription errors. If a word looks garbled but the intended concept is clear, judge the intended concept.
• If the student uploaded notes, the notes are the source of truth. Otherwise use textbook-level knowledge.
• Never invent mistakes the student didn't make. At most 5 missed concepts, so feedback stays useful instead of overwhelming.
• Score ≈ 60% accuracy + 40% coverage (partial counts as half).
• Return strict JSON. Temperature 0.2.

Then the backend doesn't trust the model blindly:

1. Gemma 4 thinks before it answers, so we strip <thought> / <think> blocks.
2. Strip code fences, take the outermost {...}, validate with Pydantic, default any missing bucket to [], clamp the score to 0–100.
3. If anything fails (network error, bad JSON), retry once on a second open-weight model, gpt-oss-120b. The UI never shows a blank screen. Errors become a banner with a Retry button.

The rubric as an Agent Skill

The grading rubric is also published as an Agent Skill: skills/recap-review/SKILL.md plus references/schema.md and references/rubric.md. Any skills-compatible agent can review a learner's explanation the same way Recap Buddy does. The backend prompts implement the same skill.

Why Does Open Innovation Matter?

1. It's what makes the business math work. A student does maybe 3–5 reviews a day. On DigitalOcean Serverless Inference, Gemma 4 31B costs a fraction of a cent per review. That's the difference between a ₹99/month plan that's profitable and one that loses money on every power user. With an open model we're not locked into one vendor's pricing either: the same Gemma weights run on DigitalOcean, Google AI Studio, or locally, and we can move whenever the price changes.

2. Student data can stay on the student's machine. A teach-back transcript is a raw recording of what someone doesn't understand yet. That's sensitive, especially for minors. With LLM_PROVIDER=ollama or lmstudio, the whole loop (notes, transcript, review, diary) runs on a laptop with no internet. A coaching centre can run it on one machine in the classroom and nothing leaves the building. A closed API can't offer that.

3. Swap, don't rewrite. The model ID is just an environment variable. When the primary model hiccups, the backend falls back to a different open-weight family (gpt-oss-120b) automatically. Next step is fine-tuning a small Gemma on our own rubric, which is only possible because the weights are open.

4. The rubric is open too. Publishing the grading rules as an Agent Skill means teachers can read exactly how students are being judged, and change it. No black-box grading.

My Agent Session

We built this in a single afternoon at Hacktoberfest Hack Day Pune × ACES with Claude Code as our AI coding assistant. We wrote a PRD first, turned it into a CLAUDE.md and task-specific skills (/recap-backend, /recap-voice, /recap-frontend, /recap-deploy-do…), and had the agent build the backend and frontend in parallel against a fixed JSON contract. The CLAUDE.md and skills are in the repo if you want to see how we steered it.

Prize Categories

• Best Use of Gemma: Gemma 4 31B is the core reviewer and diary writer. Gemma 4 26B-A4B runs via Google AI Studio, and Gemma 4 e4b runs fully offline through Ollama / LM Studio.
• Best Use of DigitalOcean: Gemma 4 and gpt-oss-120b are served through DigitalOcean Serverless Inference, with an App Platform spec (.do/app.yaml) for one-app, two-component deployment.

Team: Tejas Nalawade (@tejassantoshnalawade) and Mohit Aher (GitHub @mohitbaher).

AI disclosure: this project was built with help from Claude Code, and this post was drafted with AI assistance.

Thanks for reading! If you try it, tell me what Buddy said you missed 👀