Debugging Cross-Model Reasoning Signature Leaks in AWS Bedrock Converse

Author: Siddharth Srivastava (@sidsri14)

Context: Upstream Fix in LiteLLM PR #50073, resolving Issue #49969.

1. The Real-World Failure Mode

When building multi-model routing gateways, a common pattern is routing an initial complex user reasoning turn to a heavy thinking model (e.g. Claude 3.7 Sonnet or Claude Haiku 3.5 with extended thinking enabled), and subsequently routing cheaper follow-up turns or tool-execution loops to a lower-cost model (such as zai.glm-5.3 or Amazon Nova) via AWS Bedrock's unified Converse API.

In production, this triggered a fatal HTTP 400 Bad Request error on the very first follow-up turn:

litellm.BadRequestError: BedrockException - {
  "message": "This model doesn't support the reasoningContent.reasoningText.signature field. Remove reasoningContent.reasoningText.signature and try again."
}

Because the gateway failed to recover, client applications retried repeatedly (8+ times in seconds), exhaustively failing on every retry with "No fallback was attempted".

2. Root Cause Analysis

Under AWS Bedrock's Converse API:

1. Model-Specific Reasoning Schemas: Anthropic Claude models on Bedrock return cryptographically signed reasoning blocks (reasoningContent.reasoningText.signature), which Claude requires when replaying past thinking turns in conversation history.
2. Strict Non-Anthropic Schema Validation: Non-Anthropic models hosted on Bedrock (such as global.zai.glm-5.3 or Amazon Nova) strictly enforce their own Converse request schema. If an assistant message contains reasoningContent.reasoningText.signature, the Bedrock Converse endpoint immediately rejects the payload with HTTP 400.
3. The Transformation Blind Spot: In LiteLLM's BedrockConverseMessagesProcessor._to_reasoning_content_block, the signature parameter was extracted and appended whenever signature is not None, regardless of whether the target downstream model was Claude or a third-party model.
4. The Recovery Matcher Blind Spot: LiteLLM implements self-healing logic for Anthropic thinking blocks via is_anthropic_invalid_thinking_block_error. However, its regex matcher strictly required both "thinking" and ("invalid" or "valid string"). Bedrock's native rejection string contains neither keyword, preventing the automatic strip-and-retry handler from ever firing.

3. Minimal Offline Reproduction (Zero Credentials / Offline)

import json
from litellm.llms.bedrock.chat.converse_transformation import AmazonConverseConfig

# Simulating an assistant turn holding a signed Claude thinking block replayed into conversation
msgs = [
    {"role": "user", "content": "Analyze this dataset structure."},
    {
        "role": "assistant",
        "content": "Here is the architectural analysis.",
        "thinking_blocks": [
            {
                "type": "thinking",
                "thinking": "Validating relational foreign key constraints...",
                "signature": "SIG_HAIKU_CRYPTOGRAPHIC_PAYLOAD_EXAMPLE",
            }
        ],
    },
    {"role": "user", "content": "Now proceed with table creation."},
]

# 1. Claude Target (Preserves signature as required by Anthropic)
body_claude = AmazonConverseConfig().transform_request(
    model="global.anthropic.claude-haiku-5-5", messages=msgs, optional_params={}, litellm_params={}, headers={}
)
print("Claude Target Signature Present:", "SIG_HAIKU" in json.dumps(body_claude))
# -> True (Correct)

# 2. Non-Anthropic Target (Replays signature, causing Bedrock rejection)
body_glm = AmazonConverseConfig().transform_request(
    model="global.zai.glm-5.3", messages=msgs, optional_params={}, litellm_params={}, headers={}
)
print("GLM Target Signature Present:", "SIG_HAIKU" in json.dumps(body_glm))
# -> True on unpatched main (Bug: causes 400 rejection from AWS)

4. The Surgical Fix

Part A: Guarding Signature Serialization by Model Target

In litellm/litellm_core_utils/prompt_templates/factory.py, Bedrock already resolves whether the target model accepts redacted/signed thinking blocks via to_claude = bedrock_converse_accepts_redacted_thinking(model, base_model).

We update _to_reasoning_content_block to gate the signature inclusion:

# Before:
case {"signature": signature} if signature is not None:
    return BedrockConverseReasoningContentBlock(
        reasoningText=BedrockConverseReasoningTextBlock(
            text=BedrockConverseMessagesProcessor._thinking_text(thinking_block), signature=signature
        )
    )

# After:
case {"signature": signature} if to_claude and signature is not None:
    return BedrockConverseReasoningContentBlock(
        reasoningText=BedrockConverseReasoningTextBlock(
            text=BedrockConverseMessagesProcessor._thinking_text(thinking_block), signature=signature
        )
    )

When to_claude is False, the thinking block falls through to a signature-free reasoningText. Downstream in add_thinking_blocks_to_assistant_content, signature-free reasoning text is automatically normalized into standard assistant content text:

if reasoning_text and not reasoning_text.get("signature"):
    reasoning_text_text = reasoning_text["text"]
    if reasoning_text_text.strip():
        assistants_part = BedrockContentBlock(text=reasoning_text_text)
        assistant_parts.append(assistants_part)

This ensures the user's reasoning context is fully retained in conversation history without violating Bedrock's schema.

Part B: Self-Healing Error Recovery Matcher

In litellm/llms/anthropic/common_utils.py, we augment is_anthropic_invalid_thinking_block_error to catch Bedrock's signature rejection phrasing:

lower: Final = error_text.lower()
if "reasoningcontent.reasoningtext.signature" in lower or (
    "reasoningtext.signature" in lower and ("doesn't support" in lower or "does not support" in lower)
):
    return True

If any edge case or unlisted third-party Bedrock model triggers this 400 error, LiteLLM's existing fallback pipeline automatically catches it, strips the signature, and retries the turn cleanly.

5. Verification & Test Suite

We authored 4 targeted unit tests across two suites:

# tests/unit/llms/bedrock/chat/test_bedrock_converse_thinking_signature.py

def test_bedrock_converse_preserves_signature_for_claude():
    """Claude models must retain cryptographic reasoningText.signature."""
    body = AmazonConverseConfig().transform_request(model="global.anthropic.claude-haiku-5-5", ...)
    assert "SIG_FROM_HAIKU" in json.dumps(body)

def test_bedrock_converse_drops_signature_for_non_anthropic_models():
    """Non-Anthropic models (GLM 5.3) must strip signature and preserve text."""
    body = AmazonConverseConfig().transform_request(model="global.zai.glm-5.3", ...)
    assert "SIG_FROM_HAIKU" not in json.dumps(body)
    # Verifies thinking text was preserved as regular content text
    assert any(part.get("text") == "pondering the question" for part in assistant_msg["content"])

def test_is_anthropic_invalid_thinking_block_error_bedrock_unsupported_signature():
    """Validates regex capture of Bedrock's exact error string."""
    err = '{"message":"This model doesn't support the reasoningContent.reasoningText.signature field. Remove reasoningContent.reasoningText.signature and try again."}'
    assert is_anthropic_invalid_thinking_block_error(err) is True

Result: 100% tests passing in both local environments and upstream CI.

6. Takeaways for Multi-Model Gateway Architecture

1. Reasoning tokens are not universally portable: A reasoning block signed by one provider/family cannot be blindly forwarded as structured reasoning to another provider's runtime.
2. Preserve text over structure during cross-family handoffs: When degrading a reasoning block from a specialized container (e.g. Claude thinking) for a generic model, serialize the thinking trace into plain assistant text rather than dropping it entirely.
3. Hard schema assertions must accompany routing proxies: Don't rely solely on downstream provider error messages; validate outgoing capabilities against the routed destination's feature matrix before dispatch.