The Ariadne Thread: Why AI Fails to Follow the Path You Define

As a full-stack developer, I don't evaluate an AI-generated solution simply by checking whether the code compiles. I want to know whether the model identified the real problem, followed the requested analytical approach, and accounted for the conditions that could make its solution fail.

This distinction becomes particularly important when working with distributed systems. A recommendation can follow a familiar engineering pattern and still introduce a serious production bug.

The problem isn't necessarily that the model lacks a relevant technique. Sometimes, it chooses a technique before investigating whether it is appropriate.

The Pattern Trap: When a Familiar Solution Is Not Enough

Consider a checkout service built with TypeScript and Node.js. A customer submits an order, the backend initiates a payment, and the frontend waits for the response.

Now imagine the payment provider processes the request, but the response never reaches the application because of a network timeout.

From the client's perspective, the operation failed. From the payment provider's perspective, it may have succeeded.

That distinction creates a difficult problem: the application cannot safely assume that a timeout means the operation was never completed.

Amazon's Builders' Library explores this exact class of problem in its engineering discussion of Making retries safe with idempotent APIs. Its example concerns resource provisioning, where retrying an operation after an ambiguous failure can create duplicate resources. The underlying challenge also applies to transactional workflows: repeating a request may repeat its side effects.

A Full-Stack Scenario: The Duplicate Payment Problem

Suppose the frontend sends this request:

POST /api/checkout
Content-Type: application/json

{
  "orderId": "ord_8421",
  "amount": 149.99,
  "currency": "USD"
}

The backend sends a payment request to an external provider. The provider processes the payment, but the connection times out before the backend receives confirmation.

The frontend displays an error. The customer clicks Pay again.

If the backend treats the second request as an entirely new operation, the customer could be charged twice.

A straightforward AI-generated recommendation might be:

Add a retry mechanism with exponential backoff to handle transient network failures.

That advice isn't inherently wrong. Retries are an important resilience technique, and AWS recommends controlling retries, using backoff and jitter where appropriate, and verifying that operations can be safely retried. See the AWS Well-Architected guidance on controlling retry calls.

But in this scenario, retrying is not the first question that needs answering.

The first question is: How can the system distinguish a repeated request from a new payment operation when the result of the original request is unknown?

Imagine giving an AI assistant this instruction:

Act as a senior full-stack engineer reviewing a production payment system. Analyze the failure semantics before proposing a fix. Distinguish a client timeout from a confirmed payment failure. Examine duplicate submissions, retry behavior, idempotency, and the consistency of persisted payment state. Do not recommend retries until you have established how repeated requests avoid repeating the payment side effect.

A response that jumps directly to retry configuration has missed the central requirement, even if its code uses a sensible backoff algorithm.

The instruction defined the analytical path. The response followed a familiar pattern instead.

What the Analysis Should Actually Examine

A reliable solution requires more than adding a request header or wrapping a network call in a retry loop.

1. Give the operation a stable identity

The client or backend needs an idempotency key that identifies one logical payment attempt.

For example:

POST /api/checkout
Idempotency-Key: checkout_8421_attempt_1
Content-Type: application/json

The key must remain the same when the same logical operation is retried. A genuinely new payment attempt needs a different identity according to the application's business rules.

The backend can use that key to recognize duplicate requests and return the previously recorded result instead of blindly executing the operation again.

Generating a fresh key on every retry defeats the purpose.

2. Handle the ambiguous outcome

A timeout tells the application that it did not receive a response within the expected period. It does not prove that the payment provider rejected the payment.

The system may need to query the provider, retry using the same idempotency key where supported, or mark the operation as pending reconciliation.

The correct action depends on the payment provider's API contract and its idempotency guarantees. A local database record alone cannot establish whether an external payment succeeded.

3. Protect the operation's state transitions

Consider this simplified TypeScript example:

async function createPayment(
  orderId: string,
  idempotencyKey: string
) {
  const existing = await db.paymentRequest.findUnique({
    where: { idempotencyKey }
  });

  if (existing) {
    return existing;
  }

  return db.paymentRequest.create({
    data: {
      orderId,
      idempotencyKey,
      status: "PENDING"
    }
  });
}

This illustrates the idea of recording an operation under a stable key. It is not a complete idempotency implementation.

Two concurrent requests could both fail to find an existing record before either creates it. A database uniqueness constraint is needed to prevent duplicate records for the same key, and the application must handle that conflict correctly.

There is another, deeper issue: recording a request in the local database and charging a payment provider are separate operations. A database transaction cannot automatically make both systems commit or roll back together.

A production design must account for that boundary. Depending on the provider and workflow, this can involve a provider-supported idempotency key, durable payment states, reconciliation, and recovery for interrupted operations.

AWS discusses the importance of making the recording of an idempotency token and the associated mutations atomic within the system that owns those mutations. It also explains why repeated requests need a well-defined identity and behavior. See Making retries safe with idempotent APIs.

The important engineering question isn't simply whether the implementation contains an idempotency key. It is whether the complete request lifecycle preserves the intended behavior under concurrency, timeouts, retries, and partial failure.

The Actual Test: Did the Model Follow the Instruction?

Now return to the original instruction.

The assistant was asked to analyze ambiguous outcomes, repeated requests, idempotency, and persisted state before recommending retries.

A useful evaluation would check whether its answer:

• Distinguishes transport failure from confirmed business-operation failure.
• Preserves the identity of a logical operation across retries.
• Accounts for concurrent duplicate requests.
• Identifies the boundary between local persistence and external side effects.
• Explains how the system recovers when a request succeeds remotely but its result is not recorded locally.
• Proposes tests for repeated submissions, concurrent requests, and timeouts after the provider has processed the operation.

If the response only recommends exponential backoff, it has not completed the requested analysis.

If it adds an idempotency header but ignores concurrent requests or partial failure, it has followed part of the path while missing other essential requirements.

This gives us a more concrete way to evaluate an AI assistant. Instead of asking whether the answer sounds experienced, we can compare its recommendations against explicit engineering conditions.

The Instruction–Execution Gap

This is the distinction I want to emphasize: defining the path is not the same as following the path.

A model can recognize the words retry, timeout, and distributed system and produce a plausible recommendation based on familiar engineering practices. That recommendation might even be correct in a different context.

The failure occurs when the model does not preserve the specific analytical priorities established by the instruction.

Giving the model a persona such as "senior full-stack engineer" can help establish the intended perspective. But a job title does not guarantee a particular method of analysis. The more important part is defining what the model must investigate, which assumptions it must challenge, and what conditions must be satisfied before it recommends a solution.

Even then, instructions alone cannot guarantee correct execution. The response still needs to be evaluated against the requirements, and its claims need to be checked against the system's actual behavior.

The Ariadne Thread

In Greek mythology, Ariadne gives Theseus a thread that helps him navigate the labyrinth.

I use that image to describe an analytical framework. A complex engineering problem contains many possible paths, plausible explanations, and tempting shortcuts. The framework establishes a route through them.

In the payment example, the route begins with the meaning of a timeout, moves through the identity of a logical operation, examines concurrency and external side effects, and only then considers retry behavior.

Jumping straight to exponential backoff skips the very questions that determine whether retrying is safe.

The thread is not the instruction itself. It is the connection between the original analytical objective and the decisions made throughout the investigation.