I Benchmarked 4 SAST Tools Against 164 CVEs — Here's What Found 100% of Them

If you're building software in 2026, you probably have some form of security scanning. Bandit for Python. GitHub's built-in CodeQL. Maybe Semgrep.

But how good are these tools actually — tested against real, known vulnerabilities?

I curated 164 CVEs across 5 languages and ran 4 free SAST tools. Results:

• Ansede: 100% (164/164)
• CodeQL: 33.6%
• Bandit: 30.9% (Python only)
• Semgrep CE: 23.2%

The gap is authorization bugs — IDOR, missing auth. Most free tools analyze single functions. Detecting IDOR requires tracing HTTP routes → auth guards → database queries across function boundaries.

@app.route("/invoice/<id>")
def get_invoice(id):
    return Invoice.query.get(id)  
    # All 3 other tools: silent
    # Ansede: IDOR — any user can view any invoice

False positives on 125 clean snippets: Ansede 0. Semgrep CE: 20-60%.

Reproduce: pip install ansede-static && python -m benchmarks.one_click_compare

Full methodology: github.com/mattybellx/Ansede

What SAST tools do you use? Have you benchmarked them?