Target DEV title: ScopeShield: I Stopped Working for Free with Gemma + TabPFN (and Kept Client Data Off Closed APIs)
Tags: devchallenge, hf26challenge, ai, opensource
Challenge: Hacktoberfest Week 2: Side Hustle — closes Oct 18, 2026 11:59 PM PDT

What I Built

ScopeShield — NDA-Safe Freelance & Boutique Agency Scope-Creep Auditor and Change-Order Engine.

Freelancers and 2-10 person agencies lose 15-28% of gross revenue to unbilled scope creep:

1. Signed 12-15 page SOW says "3 revision rounds, React frontend, no DevOps."
2. Client Slacks: SSO, Hindi locale pack, K8s hotfix, "just one more revision."
3. You do it to be nice. You never invoice it. You eat $1,800 that week.

Upload SOW PDF/Markdown + weekly CSV (Jira/Linear/GitHub/Toggl) → line-by-line ledger with IN_SCOPE / GREY_AREA / OUT_OF_SCOPE + clause citations + recoverable $ + one-click change order + polite email grounded in live market rates.

Who pays? Solo freelancer Priya + micro-agency owners defending overages to client finance. $19/mo solo — viable because open weights make marginal token cost $0. One saved 4h ticket at $70/h pays 14x.

Verified offline (10/10 tests green, no keys): 53.0h logged → 26.0h out-of-scope → $1,845 recoverable across 12 rows, 4 leaks:

• JIRA-105 SSO + audit logs 11h → Section 5.2 Exclusion — $770 (leak 0.909)
• JIRA-106 Hindi locale pack 6h → Section 7.1 Exclusion — $450 (leak 0.915)
• JIRA-107 Prod K8s hotfix 5h → Section 7.1 Exclusion — $425 (leak 0.887)
• JIRA-108 Hero 5th revision 4h → Section 4.2 Revision Policy — $200

Demo

1-click judge path, no upload:

make setup && make test   # 10/10 green
make api  # uvicorn :8000 → GET /api/demo/audit
make web  # vite :5173 → Run Real Agency Audit

• GET /api/demo/audit — full pipeline on bundled SOW (14 clauses) + CSV (12 rows)
• GET /api/demo/ledger.csv — finance-ready export
• GET /api/health — tiger/tabpfn/gemma/serp status
• UI: Upload → traffic-light ledger → editable change order + Copy Email + Download + telemetry drawer

Links:

• Repo: https://github.com/Aryan1092raj/ScopeShield

Local demo is the source of truth — no hosted deploy.

GIF checklist: ledger with 4 red pills → expand JIRA-105 §5.2 quote → email preview → telemetry JSON.

Code

Repo: https://github.com/Aryan1092raj/ScopeShield (main, MIT)

On DEV embed with:

make setup          # pip + npm install
make test           # 7/7 green: 4/4 leaks, $1845, $26h OOS
make bench          # Tinker 250+50 pairs → validity/F1 table
make api            # uvicorn :8000 → GET /api/demo/audit
make web            # vite :5173 → Run Real Agency Audit

# 1. Tiger Data: run README §7.1 DDL, then:
psql $DATABASE_URL -f scripts/seed_tiger.sql
# 2. Render: New → Blueprint → point at render.yaml (api + static

scopeshield/
├── backend/main.py audit_service.py sow_parser.py db_tiger.py
│           engine_tabpfn.py engine_serp.py engine_gemma.py schemas.py test_audit.py
├── data/demo_sow_contract.md demo_weekly_git_jira.csv baseline_agency_tickets.csv
├── scripts/generate_tinker_dataset.py benchmark_tinker.py seed_tiger.sql
└── frontend/ React 19 + Vite + Tailwind ledger UI

Hybrid RRF SQL (Tiger Data, README 7.1, no LangChain):

SELECT c.section_number, c.clause_text, c.is_exclusion_clause,
  COALESCE(s.dist, 1.0)::FLOAT AS semantic_distance,
  (COALESCE(1.0/(rrf_k+s.rank_ix),0.0)+COALESCE(1.0/(rrf_k+k.rank_ix),0.0))::FLOAT AS rrf_score
FROM semantic_search s FULL OUTER JOIN keyword_search k USING (clause_id)
JOIN sow_clauses c ON c.clause_id = COALESCE(s.clause_id, k.clause_id)
ORDER BY rrf_score DESC LIMIT 3;

Local fallback match_memory() mirrors the fusion so demo works with zero infra.

How I Built It

Every sponsor tech has a non-overlapping job:

• Tiger Data (pgvector + tsvector, RRF pure SQL): SOW chunked by Section X.Y, 768-d via nomic-embed-text local via Ollama, top-3 clauses per row.
• TabPFN-3.5 (tabpfn-client): 9-feature vector per row to predict_proba to scope_leakage_probability. LLMs hallucinate math, TabPFN does not. Heuristic fallback when no key.
• Gemma 2 9B-IT (open weights via Ollama/vLLM) + Tinker LoRA: strict-JSON verdict + citation + email/order. 250+50 pairs via generate_tinker_dataset.py, benchmark_tinker.py reports validity/F1/tokens. Adapter gemma-2-9b-tinker-sow-v1. Stub judge for offline CI, Ollama path with repair retry for real Gemma.
• SerpApi: freelance {category} rate 2026 to regex median + 24h cache + fallback table (frontend 55, backend 70, devops 85, design 50, compliance 90, database 75).
• Sentry: ScopeShield_Full_Audit transaction + 4 spans (Tiger/TabPFN/SerpApi/Gemma), trace id in UI drawer. Local-first — no hosted deploy.
• Entire / Copilot: session export for appendix.

Pipeline per row (audit_service.py): index clauses to match_memory/match_tiger to build_feature_row to score_rows to fetch_market_rate to judge_row_stub/judge_row_ollama to totals + change-order markdown + client email.

Tinker honesty: dataset + harness shipped, stub simulation shows validity +25pp / F1 +0.15 / tokens -35% directionally. Full GPU LoRA next, contract frozen and reproducible.

Why Does Open Innovation Matter?

1. NDA-safe: Gemma local, embeddings in-VPC, Tiger in your Postgres. Raw SOW never leaves your boundary. Embeddings-only + auto-delete kills the you store contracts objection.
2. Zero marginal cost = 19/mo viable: local Gemma + cached TabPFN/SerpApi, tokens 0. Telemetry token_savings_vs_gpt4o_pct 35.0.
3. Fine-tune beats prompting: Tinker LoRA teaches strict JSON and exclusion enforcement (5.2/7.1 traps), swappable gemma-2-9b to gemma-3-4b via env.
4. Separation a closed LLM cannot do: TabPFN owns numbers, Tiger owns retrieval, Gemma owns words. Composable open pieces, not one black box.

If a closed model reprices or drifts, ScopeShield still audits. That is the moat.

My Agent Session

Build trace: Thu Tiger+TabPFN, Fri Gemma+Tinker+SerpApi, Sat demo polish + DEV draft. Entire/Copilot export in appendix.

Session embed pending — will add the DevRelay session link here once saved.

Prize Categories

• Best Use of TabPFN (200): engine_tabpfn.py 9-feature fit-120 predict_proba inside agent
• Best Use of Tinker (200): 250+50 pairs + benchmark validity/F1/tokens
• Best Use of Gemma (200): gemma-2-9b-it Ollama strict-JSON repair retry
• Best Use of Tiger Data (100): pgvector+tsvector RRF SQL
• Best Use of SerpApi (100): live rate grounding 24h cache cited in order
• Best Use of Sentry Agent Tracing (100): full-audit transaction + 4 spans
• Best Use of Entire (100): sessions in appendix
• Best Use of GitHub Copilot (100): agent/CLI build

By @aryan1092raj — solo build, MIT open source.