I Built an AI Agent Security Scanner. Semgrep and CodeQL Detect 0 Percent of These Attacks
I have spent the last 6 hours building what I believe is the most comprehensive AI agent security scanner in existence. The Numbers Metric Value Detection rules 18 (10 OWASP ASI + 5 novel) Benchmark
I have spent the last 6 hours building what I believe is the most comprehensive AI agent security scanner in existence.
The Numbers
| Metric | Value |
|---|---|
| Detection rules | 18 (10 OWASP ASI + 5 novel) |
| Benchmark | 50 samples (100% detection, 0 FP) |
| Tests | 96 passing |
| Frameworks scanned | LlamaIndex 252C, AutoGen 80C |
| Semgrep | 0% on same benchmark |
| CodeQL | 0% on same benchmark |
5 Novel Vectors
- Memory Poisoning - corrupting vector stores
- Tool Output Trust - blind trust in tool results
- Action Chain Amplification - single trigger mass destruction
- Multi-Agent Collusion - agents conspiring through shared state
- Prompt Template Injection - structural prompt attacks
pip install dfx-agentguard
GitHub: https://github.com/dockfixlabs/agentguard
Benchmark: https://dockfixlabs.github.io/agentguard-benchmark/
📰 Read the original article on Dev.to Security
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.