Dev.to WebDev 🛠 Dev 👁 0 📖 6 min read

An index is a cache with a schedule: Magento 2 indexers vs Laravel Scout

One index on a store I maintain had been stuck since 16 March, 17:00:34. I found it on 17 September, while deploying a security patch. For 6 months the only trace was one CRITICAL line a day in var/log/cron.log.

One index on a store I maintain had been stuck since 16 March, 17:00:34. I found it on 17 September, while deploying a security patch. For 6 months the only trace was one CRITICAL line a day in var/log/cron.log.

How an index freezes

Magento marks an index "working" when a full reindex starts and clears the mark when it ends. That mark in indexer_state is the lock, and it has no expiry and no stale detection. In March a reindex was killed at the OS level, mid-run, and the mark stayed behind. Every nightly indexer:reindex after that saw "working", assumed another process was busy, and skipped the index.

It belonged to an order-export module with no export profiles set up yet, so no exports were affected.

You can reproduce it on a fresh install in about a minute. In indexer_state, set the Stock index's status to working with one SQL UPDATE, and run bin/magento indexer:reindex cataloginventory_stock. It prints "process error during indexation process:" and then:

Stock index is locked by another reindex process. Skipping.

The command exits with 1 and the Stock index isn't rebuilt. Core cron is quieter still: its reindex job only takes indexes marked invalid. A "working" one is skipped without a line anywhere.

By hand, bin/magento indexer:reset cataloginventory_stock clears the mark and the next reindex runs.

On that store the fix was 2 things: an interrupted update now restarts on its own, and a daily check emails if any index is stuck. I watched the next 4 nightly runs. None was skipped.

The second mark

Each Magento index updates in one of 2 modes. In Update on Save, the indexer runs when Magento's own code saves the entity. In Update by Schedule, MySQL triggers write every change to a changelog table, and a cron job works through it every minute. On a fresh Magento 2.4.9 install every index is in the second mode.

That cron job has a lock of its own: a "working" status in mview_state, again with no expiry. The 2 marks fail differently. With the reindex mark frozen, changelog updates kept arriving: a quantity changed from 100 to 77 reached the stock index on the next cron pass. With the changelog mark frozen, they stopped. The stock table said 88, the index still said 77, and indexer:status showed "Ready" in its Status column. Only the Schedule Status column of the same row said "working".

A full reindex clears a frozen changelog mark. Nothing starts one for you: the index counts as valid, and cron leaves it alone.

The check is small

A check like the one below runs from cron and treats 6 hours as longer than any real reindex takes. It reports both: any index that hasn't been "valid" for that long, a deliberately suspended one included, and any changelog update that has been "working" that long.

<!-- every day at 06:00, cron calls execute() -->
<job name="acme_stuck_index_alert" instance="Acme\Ops\Cron\StuckIndexAlert" method="execute">
    <schedule>0 6 * * *</schedule>
</job>
class StuckIndexAlert
{
    // Alert is yours: a mailer or a chat hook
    public function __construct(
        private ConfigInterface $indexerConfig,
        private IndexerRegistry $indexerRegistry,
        private Alert $alert
    ) {}

    public function execute(): void
    {
        $limit = time() - 6 * 3600;
        foreach (array_keys($this->indexerConfig->getIndexers()) as $id) {
            $indexer = $this->indexerRegistry->get($id);
            // full reindex: after 6 hours, "valid" is the only healthy answer
            $state = $indexer->getState();
            if ($state->getStatus() !== StateInterface::STATUS_VALID
                && strtotime($state->getUpdated()) < $limit) {
                $this->alert->send("Index $id is {$state->getStatus()} since {$state->getUpdated()}");
            }
            // changelog updates: a view "working" for that long has stopped
            $view = $indexer->getView()->getState();
            if ($view->getStatus() === View\StateInterface::STATUS_WORKING
                && strtotime($view->getUpdated()) < $limit) {
                $this->alert->send("Changelog update of $id is working since {$view->getUpdated()}");
            }
        }
    }
}

Against each of the 2 frozen marks it fired once: "Index cataloginventory_stock is working since 2026-11-18 02:57:51" and "Changelog update of cataloginventory_stock is working since 2026-11-18 02:57:51".

Magento's own switch for the lock

Since 2.4.3, Magento has an option for the lock itself: indexer/use_application_lock in env.php. With it on, and with the default db lock provider, "working" is backed by a MySQL lock that dies with the process. The same stale reindex row then reads as "Reindex required" instead of "Processing", and the next reindex simply runs. The frozen changelog mark reads as idle, and the next cron pass brought the index up to date.

'indexer' => [
    // "working" is then backed by a MySQL lock that dies with the process,
    // so a killed reindex reads as invalid and cron runs it again
    'use_application_lock' => true,
],

That covers a killed process, for both marks. It's also why the check looks for "not valid" and not for "working": with the switch on, the frozen reindex row is reported as invalid.

What bypasses Update on Save

The 2 modes also differ when something writes to a table directly. Take one SQL UPDATE of cataloginventory_stock_item.qty. In schedule mode the trigger adds a changelog row for cron to pick up. In save mode there's no trigger: the index keeps the old quantity, and its status still says "valid".

Laravel Scout has the same gap

With Scout 11's default collection driver there's no index to go stale: a search reads the model's table. The gap opens with an external engine such as Algolia, Meilisearch or Typesense. Then the search index is a copy too, and it's only as fresh as its last sync. A Searchable model syncs from Eloquent's model events: create and save send an update to the engine, delete sends a delete.

Product::query()->update([...]) runs one SQL statement and fires no model events. With a test engine that records every call in place of the search server, create, update and delete produce 3 calls. The bulk update produces none. Calling ->searchable() on the same query afterwards re-sends the rows that query matches.

With queue on in config/scout.php, each sync becomes a queued job: MakeSearchable for a save, RemoveFromSearch for a delete. When the search server is down, the job fails and lands in failed_jobs, where you can count it and alert on it. That config also sets after_commit, Scout's switch for the problem from part 2: syncing before the transaction commits.

class Product extends Model
{
    // synced from saved() and deleted() model events
    use Searchable;

    public function toSearchableArray(): array
    {
        return ['id' => $this->id, 'sku' => $this->sku, 'name' => $this->name];
    }
}
// Product::query()->update([...]) fires no model events: the index never hears
// a failed sync lands in failed_jobs, where you can count it
'queue' => true,
'after_commit' => true,

The rule

An index is a cache with a schedule. Know how yours updates, what bypasses that, and who notices when it stops: a check, or a customer. In Magento, alert on any index that stays "not valid" for hours and on any changelog update that stays "working", and turn on the application lock. In Laravel, queue the sync, watch failed_jobs, and call searchable() after every bulk update.

Next: your queue worker remembers the old release.

What tells you an index is stale: a check, or a customer?

Part 9 of 15 of the Magento 2 vs Laravel series. Originally published on LinkedIn.

📰 Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.