Your LLM returned valid JSON. That doesn't mean it's telling the truth.

What building an automotive telemetry application taught me about structured output, evidence validation, and knowing when not to trust a generated answer.

The response looked perfect

Recently, while preparing a conference talk, I was building an application that uses a language model to explain automotive telemetry.

The idea was to collect measurements, analyze how they change over time, detect relevant anomalies, and translate the results into explanations that people could actually understand.

The architecture seemed straightforward. A TypeScript backend would process the telemetry, a language model would generate a structured explanation, and an Angular dashboard would display it.

But there was a question I couldn't ignore: what happens when the model returns something that looks perfectly correct but isn't?

Consider this observation:

{
  "text": "Coolant temperature reads 118°C.",
  "measurementClaim": {
    "sensorId": "coolant_temp",
    "value": 118,
    "unit": "°C",
    "evidenceId": "ev-meas-coolant-01"
  }
}

The JSON is valid. Every field has the expected type. The evidence reference looks legitimate.

A schema validator has no reason to complain.

There's just one problem.

The actual coolant temperature is 106.1°C.

This is a deliberately constructed example from the companion repository, but it illustrates an important problem in applications that use language models to explain real-world information.

A response can be structurally perfect and factually wrong.

And when that response appears inside a polished application, users may reasonably assume it has been verified.

That's where the interesting engineering work begins.

Why I didn't want the LLM to do everything

One of the first architectural decisions I made was to separate data analysis from natural-language explanation.

The application doesn't ask the language model to calculate temperature trends, identify threshold crossings, or estimate how quickly a measurement is changing.

Those responsibilities belong to deterministic code.

The backend processes telemetry, computes trends, evaluates configured rules, and produces structured findings. The language model receives those findings and explains them.

Why?

Because I want to be able to test the calculations independently.

If the temperature rises by a certain amount over a given interval, I should be able to reproduce the result. If the application reports a threshold crossing, I should know exactly which rule triggered it.

Of course, deterministic code can still contain bugs. It can also operate on inaccurate data.

But its behavior is inspectable and reproducible.

The language model has a different job: making those results understandable without inventing additional facts.

And that second part is harder than it sounds.

Structured output is not fact validation

Structured output is incredibly useful when integrating language models into applications.

Instead of receiving an unpredictable block of text, we can define a contract describing exactly which fields the response should contain.

In my TypeScript example, I use Zod to represent observations, inferences, suggested checks, evidence references, and limitations.

For measurements, the schema looks like this:

export const MeasurementClaimSchema = z.object({
  sensorId: z.string().min(1),
  value: z.number(),
  unit: z.string().min(1),
  evidenceId: z.string().min(1),
});

An observation can contain a human-readable description alongside a structured claim:

export const ObservationSchema = z.object({
  text: z.string().min(1),
  measurementClaim: MeasurementClaimSchema.optional(),
  anomalyClaim: AnomalyClaimSchema.optional(),
  trendClaim: TrendClaimSchema.optional(),
});

The complete response includes additional fields:

export const LlmResponseSchema = z.object({
  summary: z.string().min(1),
  observations: z.array(ObservationSchema).min(1),
  inferences: z.array(InferenceSchema),
  suggestedChecks: z.array(z.string().min(1)),
  evidenceIds: z.array(z.string().min(1)).min(1),
  limitations: z.array(z.string().min(1)).min(1),
});

This contract helps the application reject missing fields, unexpected types, and malformed structures.

But look again at the fabricated measurement:

{
  sensorId: "coolant_temp",
  value: 118,
  unit: "°C",
  evidenceId: "ev-meas-coolant-01"
}

Every field satisfies the schema.

Zod knows that value must be a number. It doesn't know what the sensor actually measured.

And that's not a limitation of Zod. It's simply not what schema validation is designed to do.

Schema validation asks: "Does this response have the expected structure?"

Evidence validation asks: "Are its checkable claims supported by the information we actually have?"

Those are two different questions.

I wanted my application to answer both.

Building an evidence package

To verify a claim, we need something to verify it against.

In the telemetry application, the backend already produces structured results. Instead of giving the language model only a collection of raw numbers, we can organize those results into an evidence package.

The package contains measurements, computed trends, detected anomalies, and information about the limits of the available data.

For the standalone tutorial, I created a synthetic telemetry scenario with a coolant temperature of 106.1°C and a rising temperature trend.

One measurement record looks like this:

{
  evidenceId: "ev-meas-coolant-01",
  sensorId: "coolant_temp",
  label: "Coolant temperature",
  value: 106.1,
  unit: "°C",
  recordedAt: "2026-03-14T10:22:41.000Z"
}

The scenario also includes a configured warning threshold of 105°C and a corresponding threshold-crossing anomaly.

An important clarification: these are synthetic demonstration data. The threshold is an illustrative heuristic, not a manufacturer specification, and the scenario does not establish a real mechanical failure.

Every relevant measurement, trend, and anomaly has an evidence identifier.

This allows a generated explanation to reference the information supporting its claims.

For example, if the model says that coolant temperature is elevated, it can reference the measurement record. If it describes a rising trend, it can reference the computed trend.

But an evidence ID alone doesn't make a statement trustworthy.

We still need to check that the reference exists and that the claim matches the referenced data.

Validating claims against evidence

The second validation layer is implemented in a function called validateEvidenceGrounding.

Its responsibility is intentionally limited: verify the structured claims that the application knows how to check.

For a measurement claim, the validator looks up the referenced measurement and compares the sensor ID, unit, and numerical value.

Here's the numerical comparison used in the repository:

const VALUE_TOLERANCE = 0.05;

if (Math.abs(measurement.value - claim.value) > VALUE_TOLERANCE) {
  errors.push({
    layer: "evidence",
    code: "FABRICATED_MEASUREMENT",
    message:
      `Claimed ${claim.sensorId}=${claim.value}${claim.unit} but evidence ` +
      `${measurement.evidenceId} has ${measurement.value}${measurement.unit}`,
    path: `${path}.value`,
  });
}

The fixed tolerance keeps the tutorial simple. In a production application, the acceptable difference would need to account for sensor precision, rounding, and domain-specific requirements.

Now consider our original example.

The response claims 118°C.

The evidence contains 106.1°C.

The values don't match, so the validator reports FABRICATED_MEASUREMENT.

The response passed schema validation, but it failed evidence validation.

And we didn't need another language model to detect the problem.

We compared a structured claim against a known measurement.

When the evidence reference doesn't exist

Another possible failure is a response that references evidence that was never provided.

For example:

{
  "evidenceIds": [
    "ev-meas-coolant-01",
    "ev-does-not-exist-999"
  ]
}

Both values are valid strings, so the schema accepts them.

But the second identifier isn't present in the evidence package.

The evidence validator rejects it with UNKNOWN_EVIDENCE_ID.

This is a small but important distinction: a plausible-looking reference is not necessarily a valid reference.

When an observation becomes a diagnosis

There's another boundary I wanted the application to respect.

An elevated coolant temperature is a measurement.

A rising temperature trend is a calculated result.

A claim that a specific mechanical component has failed is a diagnosis.

Those statements require different levels of evidence.

In the tutorial, inferences have explicit categories:

export const InferenceKindSchema = z.enum([
  "threshold_crossed",
  "trend_continuation",
  "data_gap",
  "mechanical_diagnosis",
]);

Notice that mechanical_diagnosis is allowed by the schema.

That's intentional.

We want to demonstrate that a response can be structurally valid while violating an application-level rule.

The evidence validator rejects this category:

export const FORBIDDEN_INFERENCE_KINDS:
  ReadonlySet<InferenceKind> = new Set([
    "mechanical_diagnosis",
  ]);

The resulting error is UNSUPPORTED_DIAGNOSIS.

This isn't an intelligent diagnostic engine. It's an explicit policy that prevents the application from accepting a particular category of unsupported inference.

And that's an important distinction to preserve.

The problem with validating arbitrary text

At this point, you might wonder why we don't simply extract every number from the generated text and compare it with the measurements.

That was one of the approaches I wanted to avoid.

Consider a vehicle model containing the number 911.

That number identifies the model. It isn't a sensor measurement.

Now consider two other values:

• 106.1°C
• 0.35°C/min

The first represents a temperature. The second represents a rate of temperature change.

Even though both contain numbers and refer to temperature-related information, they describe different physical quantities.

A validator that scans arbitrary text for digits can easily confuse model identifiers, measurements, rates, timestamps, and thresholds.

Instead, the tutorial validates explicit, typed claims.

The tests include a case where 911 appears in the vehicle name. The response is accepted when its structured measurement claim is correct.

But if the response claims a coolant temperature of 911°C, the validator rejects it.

This approach doesn't solve every semantic validation problem.

It deliberately avoids pretending that a few regular expressions can reliably understand arbitrary natural language.

The more precisely we represent a claim, the more precisely we can validate it.

That became one of the most useful lessons from this experiment.

What happens when validation fails?

Rejecting an invalid response is useful, but it creates another problem.

What should the application display instead?

We could show an error message.

We could ask the language model to try again.

Or we could construct a simpler explanation directly from the evidence we already have.

For this project, I chose a deterministic fallback.

The pipeline follows three validation stages:

         Model response
                |
                v
           Parse JSON
                |
                v
        Validate schema
                |
                v
        Validate evidence
                |
          +-----+-----+
          |           |
        Pass         Fail
          |           |
          v           v
        Accept    Deterministic
        response     fallback

The orchestration lives in runTrustPipeline.

If JSON parsing, schema validation, or evidence validation fails, the pipeline calls:

buildDeterministicFallback(evidence)

Instead of asking the language model to generate another explanation, the fallback uses the existing evidence package.

It can report the observed temperature, the computed trend, and the configured threshold crossing without inventing a mechanical cause.

The explanation is less flexible, but it is generated through predictable code.

This doesn't make the underlying evidence infallible. It does mean the fallback can be tested independently of the language model.

And it prevents a rejected model response from becoming the only source of information available to the user.

Making the problem reproducible

I wanted this distinction to be something developers could run and inspect, not just read about.

So I extracted a small standalone TypeScript example with Zod and Vitest.

It uses five predefined response scenarios:

The most interesting cases are B, C, and D.

All three pass schema validation.

All three fail evidence validation.

Here's the actual test for the fabricated measurement:

it("rejects a fabricated numerical claim", () => {
  const result = runTrustPipeline(
    JSON.stringify(sampleB),
    evidence,
  );

  expect(result.jsonParseSucceeded).toBe(true);
  expect(result.schemaValidationSucceeded).toBe(true);
  expect(result.evidenceValidationSucceeded).toBe(false);

  expect(
    result.errors.some(
      (e) => e.code === "FABRICATED_MEASUREMENT"
    )
  ).toBe(true);

  expect(result.explanationSource).toBe(
    "deterministic_fallback"
  );
});

This test captures the central idea of the article.

The response is valid JSON.

It matches the expected schema.

But it contains a numerical claim that conflicts with the evidence, so the application refuses to accept it.

Run the example yourself

The companion project uses TypeScript, Zod, and Vitest. It requires Node.js 20 or later.

cd 01-valid-json-is-not-truth

npm install

# Run all five scenarios
npm run demo

# Run scenario B
npm run demo:b

# Run the tests
npm test

# Check TypeScript types
npm run typecheck

The CLI reports which validation stages succeeded, which errors were detected, and whether the final explanation came from the predefined model-style response or the deterministic fallback.

The default example doesn't call a real language model.

That's intentional.

Using predefined responses makes every scenario reproducible, requires no API key, and allows us to focus on the validation logic rather than the variability of generated output.

What this approach cannot guarantee

There's one part of this experiment I think is especially important to discuss.

This validation pipeline does not guarantee that every generated statement is true.

It checks a defined subset of structured claims.

Fields such as summary, observation text, and suggestedChecks still contain natural language. The tutorial doesn't perform complete semantic verification of those fields.

A model could place an unsupported statement inside free text without expressing it as a structured claim.

Similarly, the mechanical-diagnosis rule relies on an explicit inference category. It cannot discover every possible unsupported diagnosis hidden in arbitrary prose.

The evidence package itself could also contain incorrect information.

If a sensor is mislabeled, an upstream calculation is wrong, or a threshold is inappropriate, comparing the generated response against that evidence won't automatically reveal the original problem.

And even when a calculated trend is correct, that doesn't mean a future prediction based on it will necessarily be accurate.

These are different problems requiring different safeguards.

I wouldn't describe this tutorial as a way to eliminate hallucinations.

What it demonstrates is something narrower, but still valuable:

We can define which claims our application knows how to verify, check those claims against explicit evidence, and reject responses that violate those checks.

Passing validation means that the structured claims covered by our implementation are consistent with the supplied evidence.

It doesn't mean that every sentence is guaranteed to be correct.

Understanding that difference is part of building trustworthy systems.

Final thoughts

Working on this telemetry project changed the way I think about integrating language models into software.

Initially, the exciting part was making complex measurements understandable through natural language.

But the more interesting engineering challenge was deciding which generated statements the application should actually accept.

Structured output gave us a predictable format.

The evidence package gave us concrete references.

The validation layer allowed us to check specific claims.

And the deterministic fallback gave us a predictable alternative when the model produced something unsupported.

None of these components makes a language model infallible.

But together, they create clearer boundaries between what the model says and what the application can verify.

Valid JSON tells us that our application can read the response. It doesn't tell us that the response deserves to be believed.

And that's a distinction worth designing for.

This article is part of my Engineering Trustworthy Applications series, inspired by a real-time automotive telemetry project.

The companion repository contains a standalone TypeScript example with synthetic telemetry, Zod, Vitest, five deterministic scenarios, and no required API key.

Source code: View the example on GitHub

Next in the series: The AI was right, but the UI made it look wrong - exploring how asynchronous responses and changing real-time data can make a technically correct explanation appear misleading.