This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
Drafted on DEV via DevRelay MCP: Edit on DEV

What I Built

Cloning a multi-gigabyte monorepo just to inspect one file or understand a recent change wastes gigabytes of bandwidth and minutes of developer time. I built Canopy (live at https://canopy-lazy-git.onrender.com) to reward developer laziness: instead of downloading thousands of untouched blobs upfront, it mounts repositories using 767-byte manifest slices and materializes files strictly on demand. When you ask for diff digests, commit messages, or component architecture, Canopy derives the answer once with Google Gemma, validates it through Zod, and caches it permanently in SQLite for instant warm reads.

The Problem: Monorepos Punish Curiosity

In large monorepos, traditional developer tools demand total commitment up front. You want to check what changed in the billing service? Run git clone, wait several minutes, pull down hundreds of megabytes of history, and watch your laptop fan spin.

Anthropic's Scion explored making git lazy for autonomous AI agents. Canopy makes a repository lazy for human developers, backed by an open-weight model and content-addressed local caching that you control.

Key Capabilities

• Lazy Slice-Based Filesystem (src/cli/client.ts): canopy mount fetches only the root directory manifest slice (767 bytes). Sub-directories and file blobs are resolved on demand and cached in .canopy/objects/ using their SHA-1 hashes. Subsequent reads hit local disk in 25.7 ms with 0 network bytes.
• Bilingual Repository Digest (canopy digest): Generates an authoritative two-line summary and potential risk flags for recent commit ranges (e.g. HEAD~5) in either English or Bengali (--lang bn).
• Architectural Role Explanation (canopy why <path>): Analyzes git commit history, file blame, and author distributions to explain a component's primary role and evolution patterns.
• Commit Message Synthesizer (canopy commit): Derives concise, conventional commit titles and bodies from staged diffs piped via stdin.
• Boot-Time Demo Cache Warmer (src/server/warmer.ts): Pre-warms key digests and architecture files on boot, dropping initial user latency from ~13.7s to ~4ms (a 3,400x speedup).
• Sentry GenAI Agent Tracing (src/server/sentry.ts): Instruments every LLM call across four designated agents (CanopyDigestAgent, CanopyWhyAgent, CanopyCommitAgent, CanopySummaryAgent) following OpenTelemetry semantic conventions, with strict developer privacy that scrubs client IP addresses and diff payloads.

Demo

Experience Canopy directly in your browser:

• 🌐 Live Web Application: https://canopy-lazy-git.onrender.com
• ⚡ Zero-Wait Demo Experience: Explore the interactive tree viewer, inspect code blobs with cached AI summaries, toggle between English and Bengali digests, inspect architectural why files, and view real-time GenAI trace telemetry.

• Walkthrough Video: docs/demo.webm (WebM, 1280x720, 86s, 5.40 MB)
• High-Resolution GIF: docs/demo.gif (540x304, 0.91 MB)

Code

• GitHub Repository: https://github.com/SayemR0018/canopy
• License: Permissive MIT License
• Tech Stack: Node >= 22.5.0, TypeScript, Hono, Dual SQLite Adapter (node:sqlite / better-sqlite3 with WAL mode), Google Gemma 4 (models/gemma-4-26b-a4b-it), Sentry, Docker, Render.

The Lazy Payoff: Real Benchmark Numbers

To measure real-world performance without network variance, Canopy includes a deterministic synthetic repository generator (scripts/seed.ts) producing 3,000 files across 40 directories and 200 commits in under 2 seconds.

Every number below comes directly from our 7-iteration benchmark script (scripts/benchmark.ts), audited automatically by scripts/check-claims.ts:

Baselines Explained

We benchmark Canopy against two standard baselines: Full Git Clone (with history), which measures the complete network and disk footprint of cloning the repository including all historical packfiles, commit objects, and the working tree (2.77 MiB); and Working Tree Size, which measures only the raw uncompressed bytes of the checked-out repository files on disk at HEAD without .git metadata (1.42 MiB). Canopy fundamentally restructures repository hydration: canopy mount transfers only a lightweight 767-byte root manifest slice, expanding directory slices (e.g. apps/web/src) and streaming individual file blobs strictly on demand with local content-addressed deduplication.

📊 Benchmark Reproduction & PRNG Technical Detail

The benchmark suite generates a deterministic synthetic monorepo using a Mulberry32 PRNG (seed: 1337). All benchmark iterations are committed as machine-verifiable artifacts in docs/benchmark-results.json and audited on every run via npm run check-claims.

What Went Wrong & Lessons Learned

Engineering software that bridges low-level git internals, embedded databases, and probabilistic AI models invariably uncovers sharp edges. Here are four real problems we hit and how we solved them:

1. The Native SQLite Dilemma on Windows vs Linux (D007)

We wanted SQLite in WAL mode for synchronous, zero-network caching. In production Docker containers on Linux, better-sqlite3 compiled cleanly. But on developer Windows machines without Visual Studio C++ build tools installed, npm install choked on node-gyp.

• The Fix: We architected a dual-engine database adapter (src/db/index.ts). It dynamically detects the runtime: on Node >= 22.5.0, it leverages Node's built-in node:sqlite (DatabaseSync), while falling back to better-sqlite3 when available. Both implement an identical TypeScript contract with zero API divergence.

🔧 Technical Detail: Dual SQLite Engine Adapter

// Dynamically selects node:sqlite or better-sqlite3 with standard WAL PRAGMAs
export function createDatabaseAdapter(dbPath: string): DatabaseAdapter {
  try {
    const { DatabaseSync } = require('node:sqlite');
    const db = new DatabaseSync(dbPath);
    db.exec('PRAGMA journal_mode = WAL; PRAGMA busy_timeout = 5000;');
    return new NodeSqliteAdapter(db);
  } catch {
    const Database = require('better-sqlite3');
    const db = new Database(dbPath);
    db.pragma('journal_mode = WAL');
    db.pragma('busy_timeout = 5000');
    return new BetterSqliteAdapter(db);
  }
}

2. The Platform-Locked Rollup Dependency Trap (D023)

While testing Docker builds, npm ci on Debian Linux crashed with EBADPLATFORM: Unsupported platform for @rollup/rollup-win32-x64-msvc. Running npm install on a Windows host had written Windows-specific optional native packages into package-lock.json.

• The Fix: We cleaned platform-locked devDependencies from package.json, pinned engines.node to >=22.5.0, and updated our multi-stage Docker build to execute npm prune --omit=dev.

3. Ephemeral Port Clashing Behind Reverse Proxies (D024)

We implemented per-IP rate limiting (60 req/min) to safeguard our Google AI Studio demo quota. In local testing, bursts returned HTTP 429 cleanly. But when deployed behind Render and Cloudflare, requests intermittently bypassed rate limits.

• The Investigation: Cloudflare and Render append proxy addresses to x-forwarded-for and attach ephemeral client ports (203.0.113.195:54321, :54322). Because each HTTP connection used a different ephemeral port, our IP map treated them as separate clients!
• The Fix: We overhauled src/server/app.ts to prioritize cf-connecting-ip, split multi-hop x-forwarded-for headers, and strip trailing :port suffixes before looking up rate limits.

4. Production Cold Starts: Model Bakeoff & Thought Token Stripping (Measured Before/After)

Generating deep architectural digests requires significant LLM inference. During initial live deployment testing, cold inference latency was a serious barrier:

• Measured Before: First-user cold digest latency took 13,705 ms p50 on Google AI Studio's Gemma 26B (models/gemma-4-26b-a4b-it), while Gemma 31B spiked to 24,200 ms p50 and hit a 116-second timeout with a degraded fallback. Furthermore, reasoning thought tokens enclosed in <thought>...</thought> blocks caused schema validation failures.
• The Fix: We evaluated 26B vs 31B across 5 live runs each (scripts/bakeoff.ts, documented in docs/bakeoff.md). 26B achieved 100% first-try schema validity vs 80% for 31B. We patched DerivedService with regex thought stripping (<(thought|think)>[\s\S]*?<\/\1>), and implemented a background cache warmer (src/server/warmer.ts) pre-populating English & Bengali digests and 3 key architecture files on boot.
• Measured After: First-user digest requests now hit SQLite cache in ~4 ms instead of ~13.7s (a 3,400x speedup), backed by 60s server timeouts and a visible warming status badge in the Web UI.

Why Does Open Innovation Matter?

Code is proprietary, developer bandwidth is scarce, and infrastructure is expensive. The open-source AI paradigm offers distinct advantages:

1. Vendor Sovereignty & Zero Lock-In: Canopy's LLM layer is built strictly on the standard OpenAI-compatible API format (/v1/chat/completions) without proprietary vendor SDKs. You can run Canopy against Google AI Studio's Gemma endpoint in the cloud, or point CANOPY_LLM_BASE_URL to a local Ollama instance (http://localhost:11434/v1) running on your laptop with zero code modifications.
2. Offline Resilience: When internet access drops or API quotas are exhausted, Canopy doesn't crash. Every operation is guarded by strict Zod schemas, 2 automatic retries, and deterministic offline heuristic fallbacks flagged is_degraded = 1.
3. Local Privacy: With local models via Ollama, zero lines of code or commit diffs ever leave your developer machine.

Honest Verification Status: Google AI Studio's hosted Gemma 4 model was fully verified live in production. The offline local path via Ollama is fully implemented with automated probing (npm run verify:ollama generating docs/ollama-results.json), but is currently marked unverified because the local Ollama daemon was not running on the Windows test machine during this build pass.

Sentry GenAI Agent Tracing & Privacy

Observability is essential when managing multi-agent AI systems. We integrated Sentry OpenTelemetry GenAI agent tracing across four designated agent personas:

• CanopyDigestAgent: Commit range synthesis and risk assessment
• CanopyWhyAgent: Architectural role and code evolution analysis
• CanopyCommitAgent: Conventional commit message generation
• CanopySummaryAgent: File-level responsibility extraction

Each span captures standard semantic attributes (gen_ai.system, gen_ai.request.model, prompt/completion tokens, input/output tokens, latency, cache hit/miss, retry count, and degraded status).

Privacy by Design

Developer code must remain confidential. In src/server/sentry.ts, we implemented strict privacy protections:

• Client IP addresses are scrubbed and never stored (user.ip_address: undefined).
• Raw diff payloads and file source contents are stripped from event context before envelope dispatch.

Run our automated trace demo generator to simulate and dispatch 20 real-world scenarios to Sentry:

npm run trace:demo

Note on Delivery Verification: Because client envelope dispatch is asynchronous fire-and-forget, traces must be visually confirmed by a human checking Explore > Agents in the Sentry dashboard.

My Agent Session

This project was built pair-programming with Antigravity, Google's agentic AI assistant.

Hackathon challenge eligibility and tracking were verified through the DevRelay MCP Gateway:

• We called devrelay-gateway/get_challenges via MCP, confirming that Challenge ID 81 (Hacktoberfest Open-Source AI Challenge: Week 3: Slow and Steady) is ACTIVE through 2026-10-25T06:59:00Z (see DEVRELAY_STATUS.md).

Working with an agent was a rigorous pair-programming dialogue:

• Diagnosing why Windows PowerShell requires ; instead of && for command chaining.
• Inspecting Vitest logs when file extension heuristics generated .tsx instead of .ts.
• Writing automated verification scripts (scripts/verify.ts) to validate every gate before committing.
• Establishing an honest engineering standard: every single claim in this post and the repository README is verified by scripts/check-claims.ts against docs/benchmark-results.json.

🧪 Automated 7-Gate Verification Suite

Every release passes 7 sequential verification gates:

npm run verify

1. Vitest suite (48 tests across 6 test suites)
2. Local Mock LLM server startup
3. Synthetic bare monorepo generation (3,000 files, 200 commits)
4. Canopy server boot with SQLite WAL
5. E2E lazy fetch benchmark (verifying >= 98% bandwidth savings)
6. Zero-call cache hit assertion & bad-model 2-retry heuristic degradation
7. Numerical claims audit against docs/benchmark-results.json

Prize Categories

• Best Use of Gemma ($200): Canopy leverages Google Gemma 4 (models/gemma-4-26b-a4b-it) via Google AI Studio's OpenAI-compatible endpoint as its primary reasoning engine for structured file summaries, bilingual git digests, and architectural role explanations backed by thought token stripping, Zod schema validation, a 2-retry repair loop, and boot-time cache warming (src/server/derived.ts).
• Best Use of Render ($200): Canopy is containerized with a production multi-stage Dockerfile and deployed on Render as an auto-seeding Web Service featuring an Infrastructure-as-Code Blueprint, automated REST API deployment orchestration, /healthz health monitoring, and proxy-aware per-IP rate limiting (render.yaml).
• Best Use of Sentry ($200): Canopy instruments every LLM call across four designated agent personas following OpenTelemetry GenAI semantic conventions with zero-leak IP and diff scrubbing, verifiable in Sentry under Explore > Agents (src/server/sentry.ts).
• Best Use of GitHub ($200): Canopy integrates automated GitHub Actions CI verification gates, a 10-minute production keepwarm cron, repository Copilot guidance, a PR template, and curated good first issues with explicit acceptance criteria (.github/workflows/ci.yml).