SkillSentry — The Laziest Way to Keep Your AI Agent's Skills Honest

This is a submission for the Hacktoberfest Week 3 DEV Challenge

Prize Categories: Best Use of Gemma

Every agent skill you install is a note written to your AI agent's future self: when this situation happens, behave this way. And almost nobody reads those notes before the agent does. I used to feel vaguely guilty about that. Then I realized the truly lazy move wasn't reading them faster — it was making the reading someone else's problem. Preferably a machine's. Preferably one that runs in 22 milliseconds.

That's why I built SkillSentry: a local-first security scanner for agent skills (SKILL.md files) and MCP server configs, with an optional second opinion from a Gemma 3 model running on my own laptop.

What I built — and what effort it saves

skillsentry is a TypeScript CLI with zero runtime dependencies. You point it at a directory of skills or an MCP config, and it returns a 0–100 risk score with concrete findings mapped to the OWASP LLM Top 10 (2026), AST10, and OWASP MCP Top 10 — the same vocabulary your security team already speaks.

Eleven static rules do the first pass:

• Prompt injection payloads — instructions hidden in skill descriptions that hijack agent behavior.
• Credential access — skills that peek at SSH keys, .env files, or token stores.
• Covert exfiltration — benign-looking text that ships your data to a webhook.
• Rug-pull detection — the nastiest one: it reads the git history and flags skills whose content was rewritten after trust was established.
• RCE one-liners — curl | bash, PowerShell iwr → iex, eval(base64(...)), chmod-and-stage chains.
• Typosquatting — Damerau–Levenshtein distance against ~40 popular npm/PyPI packages, because reqeusts is not requests.
• Registry overrides, base64 obfuscation, and more.

Everything is wired for the laziest possible workflow: SARIF 2.1.0 output that drops straight into GitHub Code Scanning, CI-friendly exit codes, and inline suppressions that require a written reason — so future-you never has to remember why a finding was ignored, the audit trail remembers for you.

There's also a dashboard with a drag-and-drop scan API, in case you're too lazy to even open a terminal.

The effort math: I currently run 40+ agent skills. Reading one carefully takes ~15 minutes; doing it again after every update is a task nobody on earth actually performs. SkillSentry does the whole folder in ~3.9 ms per file (benchmarked at ~2,500 files/s), 127 tests behind it, coverage gates enforced in CI.

Demo

What you're watching: a poisoned SKILL.md — the kind that politely asks an agent to exfiltrate its context — gets flagged DANGEROUS, risk 71/100, 5 findings in 22 ms. A clean skill passes clean. And notice what happens when no local model is running: the scan doesn't fail, it degrades gracefully to static-only. Laziness is a supported configuration.

Run it yourself:

git clone https://github.com/Ulk0r0/skillsentry.git
cd skillsentry && npm install && npm run build
node packages/skillsentry/dist/cli.js ./your-skills --fail-on high

Where Gemma comes in

Static rules catch known shapes. Attackers write novel shapes. That's the gap where most security tooling quietly dies — so SkillSentry has an optional semantic layer, and it's where Gemma earns its sticker.

Point it at any OpenAI-compatible endpoint running an open-weight model — gemma3:4b on Ollama is the default, and llama.cpp server, LM Studio, or vLLM work identically:

ollama pull gemma3:4b
node packages/skillsentry/dist/cli.js ./your-skills

That's the whole setup. If Ollama is listening on its default port, the semantic layer just switches on.

Gemma's job is to judge intent, not vocabulary. For every skill it returns a structured verdict: what the text is really trying to make the agent do, which manipulation techniques it uses (instruction override, covert exfiltration…), what data flows where, a severity with a confidence score, and a rationale that must cite the exact phrases that justify the verdict. The system prompt forces it to map everything onto OWASP LLM Top 10 / MCP Top 10 ids, so Gemma's opinion lands in the same report as the static rules — not in a separate "AI magic" panel.

Two design decisions I'm proud of:

1. Nothing leaves your machine. The open weights run locally via Ollama. No API key, no hosted inference, no per-scan bill, and the sensitive skill files you're auditing never travel anywhere.
2. The model is optional at every level. Probe it, and if the runtime is down, the scan continues static-only with aiUsed: false in the JSON. You can be lazy about uptime, too.

The lazy payoff

The laziest part isn't any single feature — it's that the whole thing is designed so you can set it up once and then trust the forgetting. Exit codes for CI, audit trails for suppressions, graceful degradation for missing models. The system assumes you will forget, and it doesn't mind.

Why open innovation matters here

A security auditor for an open ecosystem shouldn't be a black box — that's just another stranger asking for the same trust it's supposed to be checking. So the auditor itself is auditable: zero runtime dependencies means the entire behavior is readable source; MIT-licensed; local-first by architecture, not by policy. You can verify every claim in this post from the repo.

And the semantic layer only works because Gemma 3 is an open-weight model. A closed API would mean shipping every skill file you audit to a third party — a data-leak dressed up as a security product. Open weights turn intent-level analysis into something you can run offline, for free, on hardware you own. That's not a convenience; it's what makes private security review of agent skills possible at all.

Finally, the standards are open too — OWASP LLM Top 10 (2026), OWASP MCP guidance, the agentskills.io spec. SkillSentry cites them instead of inventing a proprietary severity alphabet, so its output plugs into conversations your team is already having.

The fine print

• Original work: built from scratch on October 20–21, 2026 — inside this challenge's window. The git history starts at the initial commit; no fork parent, no code reused from prior projects. Published standards are cited, never copied.
• Built with an AI coding agent, in one marathon pair session. Given that this month's theme is open-source AI, building the security scanner with the kind of agent it protects felt appropriately self-referential.
• Repo: github.com/Ulk0r0/skillsentry — v0.4.0, MIT. Clone it, break it, tell me what the rules missed.