Dev.to Security 🔐 Cybersecurity 👁 1

Crucible: Pytest for AI Agents

One thing becoming increasingly obvious while working with AI systems: Testing functionality is not enough. An AI agent may: • complete tasks correctly • respond fluently • appear fully functional …while still fail

One thing becoming increasingly obvious while working with AI systems:

Testing functionality is not enough.

An AI agent may:

• complete tasks correctly

• respond fluently

• appear fully functional

…while still failing under adversarial or unexpected inputs.

That becomes dangerous once agents gain:

  • memory
  • tool access
  • autonomy
  • long-running execution

Traditional software testing doesn’t fully cover behavioral failures in AI systems.

So we started building Crucible:

An open-source security testing framework for AI agents.

Core capabilities:

• adversarial testing

• prompt injection evaluation

• behavorial monitoring

• security scanning for agentic systems

Simple setup:

**pip install crucible-security

crucible scan --target https://your-agent-endpoint.com
**

Built with:

Python • FastAPI • LangChain • Pydantic

Open-source under Apache 2.0.

GitHub:

https://github.com/crucible-security/crucible

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.