Dev.to AI πŸ€– Ai πŸ‘ 0 πŸ“– 10 min read

Building a news pipeline with the Laravel AI SDK: agents, structured output and the pitfalls nobody warns you about

NewSumm reads news from hundreds of outlets, groups articles about the same event into one story, and writes a short summary of what the outlets agree on and where they differ. On a normal day that is tens of thousands o

NewSumm reads news from hundreds of outlets, groups articles about the same event into one story, and writes a short summary of what the outlets agree on and where they differ. On a normal day that is tens of thousands of articles.

The whole pipeline runs on the Laravel AI SDK (laravel/ai): embeddings for every article, and agents that summarise stories, describe each outlet's angle and extract the people and places a story is about. This post is about the Laravel side: how the code is organised, what the SDK gives you for free, and the three failures that cost us real time because no test could see them.

If you want the maths behind the grouping (embeddings, running centroids, how the clusters are distributed), that is in Clustering the news with cheap models. Here we stay in PHP.

The code below is simplified from our real classes. The prompts, models and thresholds are left out on purpose; the structure is what transfers.

1. The shape of the pipeline

Every step is a queued job:

  1. Fetch feeds and store new articles.
  2. Embed each article: one vector from its title and description.
  3. Cluster articles into stories by vector similarity, in Postgres with pgvector.
  4. Summarise each story that more than one article has joined.

Only step 4 calls a language model, and only for multi-article stories. In a two-day sample of 51,842 articles, 4,748 stories had two or more articles: 9.2% of the article count. Most of the cost decision is made before any agent runs: send the model stories, not articles.

2. Embeddings: batch them, and guard the input

The SDK's embeddings API is one line:

use Laravel\Ai\Embeddings;

$response = Embeddings::for($texts)->generate();

$vectors = $response->embeddings; // one array of floats per input, same order

We embed in chunks of 50 so one HTTP request covers 50 articles. The call itself was never the problem. The input was.

Feed data is messy, and a single bad string makes the provider return a 400 for the whole batch. We hit three:

  • Too long. Some feeds put the full article in the description. We cap the text well under the provider's token limit, and use a lower cap than the usual "four characters per token" rule suggests, because dense non-Latin scripts use far more tokens per character.
  • Empty. No title, no description. Skip it.
  • Looks empty, isn't. Titles made only of a non-breaking space or a zero-width space. trim() and filled() both let these through, so the provider sees an empty string and rejects the batch.
private function isBlank(string $text): bool
{
    return $text === '' || preg_match('/^[\p{C}\p{Z}]+$/u', $text) === 1;
}

The third one hurt most. A rejected batch left its articles marked unprocessed, so they were retried on every run, and the job (then oldest-first) never got past them. Fresh articles stopped being embedded, so nothing clustered, so nothing was summarised. The site simply stopped showing new stories. Two fixes: guard the input, and process newest first, so one poisoned article can never hold up today's news.

Tip

When a batch call can fail on a single item, mark items you can't process as done, with no result rather than leaving them in the queue. Retrying a bad input forever is a slow outage.

3. Agents are just classes

An agent in the SDK is a class that implements Agent and uses the Promptable trait. Instructions live in a method, so they're versioned, reviewed and tested like any other code:

namespace App\Ai\Agents;

use Laravel\Ai\Attributes\UseCheapestModel;
use Laravel\Ai\Contracts\Agent;
use Laravel\Ai\Promptable;
use Stringable;

#[UseCheapestModel]
class StorySummaryAgent implements Agent
{
    use Promptable;

    public function __construct(public bool $compareOutlets = false) {}

    public function instructions(): Stringable|string
    {
        return 'You are a news editor. Summarise what these articles agree on...';
    }
}

Two things here earned their place.

#[UseCheapestModel] tells the SDK to use the provider's cheapest text model. Our tasks are narrow (summarise, compare, fill a schema), and small models do them well. The attribute keeps model names out of our code: switch provider, and every agent follows.

Constructor arguments flow through make(), so one agent can vary its behaviour per call without a second class:

$response = StorySummaryAgent::make(compareOutlets: $story->hasEnoughSources())
    ->prompt($this->articlesPrompt($articles));

When compareOutlets is on, the agent's instructions and schema both gain a "where coverage differs" section. One class, one test file, two behaviours.

4. Structured output turns a writer into an extractor

Free text is hard to store. We want a headline, a summary, key points, a category from a fixed list, one angle per outlet, and the people and places involved. Implement HasStructuredOutput and describe the shape:

use Illuminate\Contracts\JsonSchema\JsonSchema;
use Laravel\Ai\Contracts\HasStructuredOutput;

class StorySummaryAgent implements Agent, HasStructuredOutput
{
    // ...

    public function schema(JsonSchema $schema): array
    {
        return [
            'headline' => $schema->string()->required(),
            'summary' => $schema->string()->required(),
            'key_points' => $schema->array()->items($schema->string())->required(),
            'category' => $schema->string()->enum(Category::values())->required(),
            ...($this->compareOutlets ? [
                'coverage_differences' => $schema->string()->required(),
            ] : []),
        ];
    }
}

The response reads like an array:

$story->update([
    'headline' => $response['headline'],
    'summary' => $response['summary'],
    'category' => $response['category'],
]);

An enum on the category means the model picks from our list instead of inventing "Tech & Science" one day and "Technology" the next. We still validate what comes back (a schema constrains shape, not truth), but the classes of garbage shrink a lot.

Share rules and schema between agents

Entity extraction came later than summaries. We didn't want to re-summarise thousands of published stories just to get their people and places, so extraction has its own agent for backfilling, and the same rules and fields are spread into the summary agent so new stories get both in one call:

class EntityExtractionAgent implements Agent, HasStructuredOutput
{
    use Promptable;

    public const Rules = <<<'PROMPT'
        Record who the story is about, and where it happened...
        PROMPT;

    public function schema(JsonSchema $schema): array
    {
        return self::fields($schema);
    }

    public static function fields(JsonSchema $schema): array
    {
        return ['entities' => /* people + places */];
    }
}

// In StorySummaryAgent::schema():
...EntityExtractionAgent::fields($schema),

One source of truth for the rules, two agents using it, and backfills never touch a published summary.

5. The strict-schema trap

This one put our daily recap out for a day, and the whole test suite stayed green.

Our provider's structured-output mode is strict: every object in the schema must say additionalProperties: false and list all its properties as required. The SDK sends requests in strict mode and adds additionalProperties: false to the root object for you. It does not add it to nested objects.

So this works:

'key_points' => $schema->array()->items($schema->string()),

and this is rejected with a 400 on every single call:

'beats' => $schema->array()->items($schema->object([
    'title' => $schema->string()->required(),
    'text' => $schema->string()->required(),
])),

The fix is one method call on each nested object:

'beats' => $schema->array()->items($schema->object([
    'title' => $schema->string()->required(),
    'text' => $schema->string()->required(),
])->withoutAdditionalProperties()),

Three things made it worse than it needed to be:

  • Fakes don't validate schemas. Agent::fake() never talks to the provider, so it can't apply the provider's rules. Every test passed.
  • The job swallowed the error. It caught the exception, logged a warning, skipped the category and published an empty recap. Nothing reached failed_jobs; it looked like a quiet news day.
  • The log hid the reason. The exception message was truncated before the part of the response body that named the bad field.

Now a test compiles every agent's real schema, exactly as the SDK will send it, and walks it recursively:

use Illuminate\JsonSchema\JsonSchemaTypeFactory;
use Laravel\Ai\ObjectSchema;

function assertStrictModeCompatible(array $schema, string $path = 'root'): void
{
    expect($schema['additionalProperties'] ?? null)
        ->toBeFalse("Object at {$path} is missing additionalProperties: false");

    expect($schema['required'] ?? [])
        ->toEqualCanonicalizing(array_keys($schema['properties'] ?? []), "Object at {$path} must mark every property required");

    foreach ($schema['properties'] ?? [] as $name => $property) {
        if (($property['type'] ?? null) === 'object') {
            assertStrictModeCompatible($property, "{$path}.{$name}");
        }

        if (($property['type'] ?? null) === 'array' && ($property['items']['type'] ?? null) === 'object') {
            assertStrictModeCompatible($property['items'], "{$path}.{$name}[]");
        }
    }
}

it('compiles a schema strict mode accepts', function (string $agent) {
    $schema = (new ObjectSchema((new $agent)->schema(new JsonSchemaTypeFactory)))->toSchema();

    assertStrictModeCompatible($schema);
})->with([
    StorySummaryAgent::class,
    EntityExtractionAgent::class,
    DailyRecapAgent::class,
]);

No API call, runs in milliseconds, and it fails the build the moment someone adds a nested object without the fix.

Warning

If a job calls a model in a loop and catches errors per item, alert when every item fails. All items failing within seconds is a rejected request, not a bad generation.

6. Agents in queues: mind retry_after

Summaries run in queued jobs, and a model call can take a while. If a job runs longer than the queue connection's retry_after, Laravel hands it to a second worker while the first is still running, and you pay for the same summary twice. Ours shipped with the default retry_after of 90 seconds against job timeouts of several minutes. That produced 726 failed jobs and a lot of duplicate model calls before we spotted it.

Rule of thumb: retry_after must be longer than the longest job timeout on that connection. We wrote that one up in full: Your Laravel jobs are running twice: retry_after vs timeout.

7. Knowing what it costs

Provider dashboards tell you what you spent. They don't tell you which agent spent it. The SDK fires an event after every call, with the token usage attached, so a listener can record each one:

use Laravel\Ai\Events\AgentPrompted;
use Laravel\Ai\Events\EmbeddingsGenerated;

class RecordAiUsage
{
    public function handleAgentPrompted(AgentPrompted $event): void
    {
        $usage = $event->response->usage;

        $this->record([
            'agent' => class_basename($event->prompt->agent),
            'model' => $event->prompt->model,
            'input_tokens' => $usage->promptTokens,
            'cached_input_tokens' => $usage->cacheReadInputTokens,
            'output_tokens' => $usage->completionTokens,
        ]);
    }

    public function handleEmbeddingsGenerated(EmbeddingsGenerated $event): void
    {
        $this->record([
            'agent' => 'embeddings',
            'model' => $event->model,
            'input_tokens' => $event->response->tokens,
        ]);
    }

    private function record(array $attributes): void
    {
        try {
            AiUsage::query()->create($attributes);
        } catch (Throwable $e) {
            Log::warning("Could not record AI usage: {$e->getMessage()}");
        }
    }
}

Laravel's event discovery picks up the handle* methods by their type hints, so there is nothing to register. Note the try/catch: bookkeeping must never fail the call it records. A summary you have already paid for is worth more than its usage row.

On top of that table sits an Artisan command, php artisan ai:usage, that groups tokens by agent (or by hour, with --hourly) and multiplies them by per-model prices from config.

8. Testing with fakes, and their limits

For everything that isn't the provider contract, the SDK's fakes are excellent:

it('stores the overview the agent writes', function () {
    EntityOverviewAgent::fake(fn (): array => ['overview' => "First paragraph.\n\nSecond paragraph."]);

    // ... run the job ...

    EntityOverviewAgent::assertPrompted(fn ($prompt) => $prompt->contains('Kyiv'));
});

it('skips articles with blank titles', function () {
    Embeddings::fake();

    // ... embed a batch containing "\u{00A0}" as a title ...

    Embeddings::assertGenerated(fn ($prompt) => count($prompt->inputs) === 49);
});

Fakes answer "does my code do the right thing with a response?". They can't answer "will the provider accept my request?". For that you need either a contract test like the schema walker above, or one small real call in a scheduled smoke check. We have the first and wish we'd had it sooner.

9. What we'd tell someone starting today

  • Decide what the model reads before choosing a model. Sending stories instead of articles cut our calls by an order of magnitude. No model choice comes close.
  • Make agents classes, with their instructions in code. They get reviewed, diffed and tested like everything else.
  • Use structured output for anything you store, with enums wherever the answer comes from a list.
  • Test the schema you send, not just the response you fake. Nested objects need withoutAdditionalProperties().
  • Guard embedding input against long, empty and invisible-whitespace text, and never let one bad item block a batch forever.
  • Record tokens per agent from day one. It's one listener, and it tells you where the money actually goes.
  • Never swallow a model error silently. Log the response body, and alert when every item fails.

The SDK makes the happy path short. Most of the work was in the unhappy paths, and those are the parts worth writing down.

πŸ“° Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.