Crucible: Pytest for AI Agents
One thing becoming increasingly obvious while working with AI systems: Testing functionality is not enough. An AI agent may: • complete tasks correctly • respond fluently • appear fully functional …while still fail
One thing becoming increasingly obvious while working with AI systems:
Testing functionality is not enough.
An AI agent may:
• complete tasks correctly
• respond fluently
• appear fully functional
…while still failing under adversarial or unexpected inputs.
That becomes dangerous once agents gain:
- memory
- tool access
- autonomy
- long-running execution
Traditional software testing doesn’t fully cover behavioral failures in AI systems.
So we started building Crucible:
An open-source security testing framework for AI agents.
Core capabilities:
• adversarial testing
• prompt injection evaluation
• behavorial monitoring
• security scanning for agentic systems
Simple setup:
**pip install crucible-security
crucible scan --target https://your-agent-endpoint.com
**
Built with:
Python • FastAPI • LangChain • Pydantic
Open-source under Apache 2.0.
GitHub:
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.