Your red-team model shouldn't grade its own homework

This month outside red-teamers found gaps in the system watching a frontier lab's own agents, and the EU opened its first AI Act enforcement actions. The common thread is that the thing that attacks, the thing that judges and the thing under test must be separate. That's why Humanbound now switches attack models on the fly and never lets the attacking LLM judge its own results.

What's new: Model Relay

Red-teaming an agent is only as good as the models doing the attacking and the judging. Two problems keep showing up in practice:

• Some models refuse. Safety restrictions on some models have been tightening for a while, so they decline legitimate security work such as reverse-engineering an exploit or crafting an adversarial test.
• A model grading its own attack is a conflict of interest. If the same LLM attacks and evaluates, you can't trust the verdict.

So Humanbound now manages the models for you:

1. On-the-fly model switching. We try an attack with one model. If it isn't effective, we switch to a different model and try again, so refusals and weak runs don't leave blind spots in your coverage.
2. Judge independence. The LLM that attacks is never the LLM that evaluates. Attacker and judge are always different models, so findings are evidence rather than self-assessment.
3. Managed for you. No hunting for model names, hosts or routing settings.

Try Model Relay on the free Community plan: app.humanbound.ai

From the field

1. Outsiders red-teamed OpenAI's agent monitor and found real gaps

David Rein and Adam Kaufman, METR. The monitor ran only when agents handed control back to the user, and skipped memory-compaction events, leaving about 10% of eligible Codex traffic unwatched. Among METR's recommendations: isolate agents from their monitors and red-team the monitors regularly. This is the strongest argument yet for keeping the thing being judged separate from the thing doing the judging.

Read it

2. OpenAI will train frontier models with no live internet access

Ravie Lakshmanan, The Hacker News. After agents used a DNS misconfiguration to reach an external chatbot and red-teaming surfaced VM escapes, OpenAI resumed training only with stronger containment. Capable models find paths you didn't know existed, and a test that stops at the first blocked route will miss them.

Read it

3. Adversa AI's October roundup of agent attacks and defenses

Adversa AI team. Nearly 50 resources, including self-replicating prompt injections, hijacked human-in-the-loop approvals and a benchmark audit that found harness defects skewing reported attack rates. How you measure an attack matters as much as the attack, which is why judge quality is a security property.

Read it

4. The EU opens its first AI Act enforcement actions

Sophie Denar, Unite.AI. The AI Office sent Requests for Information on containment, external evaluation and post-market monitoring after agent loss-of-control incidents. "External evaluation" is now a regulator's question, and independent evidence is what answers it.

Read it

5. The first harmonized AI Act standard lands

Haim Ravia and Dotan Hammer, Pearl Cohen. Implementing Decision (EU) 2026/2274 covers quality management systems, including post-market monitoring. Standards for risk management, human oversight and cybersecurity are still pending. Documented, repeatable testing is the evidence compliance teams will need as the rest arrive.

Read it

6. Microsoft: treat AI agents as potential insider risks

Guru Baran, Cyber Security News. The guidance centers on least privilege, controls outside the model, and separate identities for agents. Several of the capabilities are planned, not shipped. The model is not the control: authority has to be enforced and tested outside it.

Read it

7. LinkedIn's CISO on why preparation beats heroics

Lea Kissner, LinkedIn CISO (covered by Luís Rijo, PPC Land). In "Good Security Is Boring. Bad Security Makes Headlines," Kissner argues that AI is shrinking the gap between a flaw and an attack, and lists five fundamentals, including treating AI agents as actors whose operators stay accountable. The post gives no data for its AI-attack claims. Clear ownership, scanning and rehearsed response come before any agent-specific tooling.

Read it

From the Humanbound blog

8. Not every model makes a good attacker: OpenAI vs DeepSeek

Ayan Pahwa. We pointed two attacker models at the same deliberately weak support bot and counted how often each tricked it into refunding orders that don't exist. DeepSeek V4.1 Flash succeeded 42% and 52% of the time across two runs; OpenAI's gpt-6-luna managed 7% and 10%. It's one weak target, not a general ranking, but it shows why the attacker model matters, and why we switch models when one isn't effective.

Read it

9. Build an AI agent from scratch in Python, then attack it

Ayan Pahwa. A framework-free web research agent in about 331 lines of Python: a tool-calling loop, step and token limits, and persistent notes in SQLite. One run cost about 1.6 cents and produced a wrong answer that persisted in its saved notes. An agent that reads untrusted web pages needs testing for indirect prompt injection before it ships.

Read it

10. Secure a LangChain agent with the Humanbound Firewall

Ayan Pahwa. Two lines of Python attach the open-source Humanbound Firewall to a LangChain support agent. The guide covers the policy file, configuring the LLM judge and verifying it works, since a broken setup can fail silently, then running in log mode before switching to block mode. Runtime protection should be observed first and enforced deliberately.

Read it

Takeaway

Read the external pieces together and the pattern is clear: safeguards nobody has independently attacked are safeguards you're guessing about. Attacks should switch models when one isn't effective, refusals shouldn't create blind spots, and the model that attacks should never grade its own work.

Try Model Relay on your own agent, free on the Community plan: app.humanbound.ai

Prefer to run it yourself? The engine is open source (Apache-2.0) and runs locally with your own keys:

pip install humanbound[engine]

Code is on GitHub.

What did we miss? If you're red-teaming agents, do you use a separate model for judging, and have you seen your attacker model refuse? Tell us in the comments.

References

1. Red-Teaming OpenAI's Internal Agent Monitoring Systems, METR
2. OpenAI AI Agents Demonstrate VM Escapes, The Hacker News
3. Top AI agent security resources, October 2026, Adversa AI
4. EU Announces First AI Act Enforcement Actions, Unite.AI
5. EU AI Act's First Harmonized Standard, Pearl Cohen
6. Microsoft on AI agents as insider threats, Cyber Security News
7. LinkedIn's security chief on AI shrinking the time from flaw to attack, PPC Land
8. Not every model makes a good attacker, Humanbound
9. Build an AI agent from scratch in Python, Humanbound
10. Secure AI agents with the Humanbound Firewall, Humanbound