The model you pick isn't always the model that answers
Fable 5.2's system card has a stat worth knowing if you ship agents: on Gray Swan's indirect prompt injection benchmark, attack success within 15 tries was 4.09%, about 4x Fable 5.1 (1.02%). Most of that gap comes from
Fable 5.2's system card has a stat worth knowing if you ship agents: on Gray Swan's indirect prompt injection benchmark, attack success within 15 tries was 4.09%, about 4x Fable 5.1 (1.02%).
Most of that gap comes from the fallback. When a message gets flagged, Fable 5.2 can hand the turn to Opus 4.8. Under attack that happened on 53% of attempts, and attacks succeeded 5.17% of the time when Opus 4.8 answered vs 2.05% when Fable answered itself.
What I'd do about it:
- Log
model,stop_reasonandusage.iterationson every request. Fallback routing is sticky for about an hour, and later turns only show afallback_messageentry inusage.iterations, so watching for the fallback block alone undercounts. - Turn off "Switch models when a message is flagged" for agents that read untrusted content and can act irreversibly (send, push, pay).
- Put an action check in front of irreversible tools, so an attack has to beat two layers.
The absolute numbers are still low. The point is that the one that answers sets the risk, and a setting built to cut refusals can route you to an older model without telling you.
Full breakdown, with the logger and the weekly threshold I use to decide when to turn the switch off: https://pub.towardsai.net/fable-5-2-is-4x-easier-to-hijack-a107805488d9?sk=a4a84fa65eabbcbe9c6ced01ec59e537
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.