Add Lucene-style search and highlighting to MUI DataGrid

MUI's DataGrid ships a filter panel: pick a column, pick an operator, type a value. It works, and it stops working the moment someone wants role:engineer AND joined:>=2021-01-01, or a wildcard, or two conditions on the same column.

The usual next step is a search box that calls Array.prototype.filter with String.includes. That gets you querying. It does not get you highlighting, and the moment you try to add it you hit a subtle problem worth understanding before we write any code.

The bug in every "highlight the search term" implementation

The obvious approach: filter the rows, then in each cell search the text again for the term and wrap it in <mark>.

That is two independent matches, and they disagree wherever their rules differ. Case folding is where it shows up first. A case-insensitive regex /s/iu marks ſ (long s) that a plain toLowerCase() comparison never matched. Drop the u flag to fix that, and now /k/i refuses the Kelvin sign K that your filter did match. You end up with rows on screen that have nothing highlighted, and highlights on rows that should not be there.

The fix is to stop matching twice. If the engine that decided the row matched also tells you where it matched, there is nothing to re-derive and nothing to disagree about.

That is what we are building, with two packages:

• @siftql/core, a Lucene-style query language for plain JavaScript objects with zero runtime dependencies
• @siftql/react-highlighter, which turns those offsets into <mark> elements

Here is what we are building. The query name:*a* filters the grid, and every character that caused the match is marked, using the offsets the filter itself produced:

There is also a live demo if you want to click around before reading further.

npm install @siftql/core @siftql/react-highlighter

Step 1: parse once, use twice

This is the whole architecture in one idea. Parse the query into an AST, then hand that same AST to both the filter and the highlighter.

import {
  parse,
  filter,
  isSiftQLError,
  type SiftQLAst,
} from '@siftql/core';

interface Parsed {
  ast: SiftQLAst | null;
  error: string | null;
}

const parsed = useMemo((): Parsed => {
  const trimmed = query.trim();
  if (trimmed === '') return { ast: null, error: null };

  try {
    return { ast: parse(trimmed), error: null };
  } catch (err) {
    const error = isSiftQLError(err)
      ? err.message
      : String(err);
    return { ast: null, error };
  }
}, [query]);

const rows = useMemo(
  () =>
    parsed.ast === null ? ROWS : filter(parsed.ast, ROWS),
  [parsed.ast],
);

Parsing inside a useMemo matters more than it looks. A search box is mid-keystroke most of the time, so parse runs on every character, and role: is a syntax error until you finish typing the value. Catching it and showing the message beats letting it throw through your render.

Step 2: index the highlights per row

Now the part most implementations get wrong.

highlight(ast, row) returns which fields matched and at what offsets. The tempting thing is to call it inside renderCell, so each cell asks about itself. Do not. highlight() walks the entire record regardless of which field you are about to render, so calling it per cell re-matches the same row once for every column.

I measured it on a 500 row, 12 column grid:

highlight() per cell : 90.5 ms   (6000 calls)
indexed once per row :  7.3 ms   (500 calls)
ratio                : 12.4x

The penalty tracks your column count, not your row count: a 20 column grid pays 20x. And 90 ms lands on every keystroke, which is a visibly janky search box.

So call it once per row and index the result:

import { highlight, type SiftQLAst } from '@siftql/core';
import type {
  HighlightSpan,
  Spans,
} from '@siftql/react-highlighter';

type FieldSpans = readonly HighlightSpan[] | null;

export interface QueryHighlights {
  spansFor(rowId: string | number, field: string): Spans;
}

/** Stable identity, so memoised columns stay memoised. */
export const NO_HIGHLIGHTS: QueryHighlights = {
  spansFor: () => undefined,
};

export const indexHighlights = <
  R extends { id: string | number },
>(
  ast: SiftQLAst,
  rows: readonly R[],
): QueryHighlights => {
  const byRow = new Map<string, Map<string, FieldSpans>>();

  for (const row of rows) {
    const byField = new Map<string, FieldSpans>();

    for (const hit of highlight(ast, row)) {
      const field = hit.segments[0];
      if (typeof field !== 'string') continue;

      byField.set(field, hit.ranges ?? null);
    }

    byRow.set(String(row.id), byField);
  }

  return {
    spansFor: (rowId, field) =>
      byRow.get(String(rowId))?.get(field),
  };
};

Two details in there are worth calling out.

Key by row.id, not by array position. The grid reorders on sort, and position stops meaning anything the moment it does.

Look up the field with hit.segments[0], not hit.path. path is a dotted display string and is lossy when a key contains a dot. segments is the canonical array form.

Step 3: the three states

hit.ranges ?? null above is the one semantic move in this whole integration, and it deserves a section.

There are three outcomes for a given cell, not two:

The middle one is the case people forget. Query joined:>=2021-01-01 and every returned row matched on that column, but there is no substring to underline. The whole value is the answer, and it renders as an underline rather than as marks:

Notice there is not a single <mark> in that screenshot. Nothing was marked because nothing should be: no substring of 2021-03-01 is the reason the row is on screen.

siftql signals this by omitting ranges, and a missed Map.get also returns undefined. So one of them has to become null on the way to the component, which is exactly what hit.ranges ?? null is doing.

Step 4: the cell renderer

import { useTheme } from '@mui/material';
import type { GridRenderCellParams } from '@mui/x-data-grid';
import { SiftQLHighlight } from '@siftql/react-highlighter';

import type { QueryHighlights } from './queryHighlights';

const ACCENT = { light: '#8D3B00', dark: '#FFA726' };

export const highlightCell = (
  field: string,
  highlights: QueryHighlights,
) => {
  const HighlightCell = (params: GridRenderCellParams) => {
    const mode = useTheme().palette.mode;
    const accent = ACCENT[mode];
    const row = params.row as Record<string, unknown>;
    const id = row.id as string | number;

    return (
      <SiftQLHighlight
        text={String(row[field] ?? '')}
        spans={highlights.spansFor(id, field)}
        markStyle={{
          backgroundColor: accent,
          color: mode === 'dark' ? '#000' : '#FFF',
          padding: 0,
        }}
        wholeValueStyle={{
          borderBottom: `2px solid ${accent}`,
        }}
      />
    );
  };

  HighlightCell.displayName = `HighlightCell(${field})`;
  return HighlightCell;
};

And wire it into your columns:

const columns = useMemo(
  (): GridColDef[] => [
    {
      field: 'name',
      headerName: 'Name',
      flex: 1,
      renderCell: highlightCell('name', highlights),
    },
    {
      field: 'email',
      headerName: 'Email',
      flex: 1,
      renderCell: highlightCell('email', highlights),
    },
  ],
  [highlights],
);

That is the integration. The rest of this post is the four things that will bite you.

Gotcha 1: never use params.value on a date column

This one cost me real time.

A type: 'date' column needs a valueGetter so MUI receives a real Date:

{
  field: 'joined',
  headerName: 'Joined',
  flex: 1,
  type: 'date',
  valueGetter: (value: string) => new Date(value),
  renderCell: highlightCell('joined', highlights),
}

Now params.value is a Date, not your string. But siftql's offsets are indexes into the original string. Slice the Date instead and you get this:

// siftql said: mark [0, 4), the year
'2021-03-01'.slice(0, 4);
// "2021"

String(new Date('2021-03-01')).slice(0, 4);
// "Sun "

Look closely at that second line. There is a second bug hiding inside it. new Date('2021-03-01') parses as UTC midnight and stringifies in local time, so at any negative UTC offset it renders as Feb 28. You would be highlighting the wrong characters of the wrong date.

So read the raw field off the row, not off params.value:

const row = params.row as Record<string, unknown>;

<SiftQLHighlight text={String(row[field] ?? '')} ... />

Gotcha 2: warning.main with getContrastText fails WCAG

The obvious way to theme the mark is MUI's warning palette with an automatic contrast foreground. It does not pass.

MUI's contrastThreshold defaults to 3, and 3:1 is the WCAG AA bar for large text. A grid cell is body text, which needs 4.5:1 for AA and 7:1 for AAA. getContrastText picks white in both light and dark mode, giving roughly 3.08:1 and 3.79:1, under AA in both.

Use dedicated tokens and measure them. Note also that light and dark need different accents, not one accent with two foregrounds: a dark mode amber sits at about 1.86:1 on a light paper surface, and a light mode brown at about 2.25:1 on dark. Both are effectively invisible.

Gotcha 3: <mark> carries user-agent padding

Browsers give <mark> a small horizontal padding. In prose you would never notice. In a dense grid a marked word sits wider than the text beside it and the column looks subtly misaligned, which reads as a spacing bug rather than as a highlight.

markStyle={{ backgroundColor: accent, padding: 0 }}

Gotcha 4: do not tint the whole cell for the null state

It is tempting to render "matched, nothing to point at" as a background tint on the cell. Try that with joined:>=2021-01-01 and every visible cell in the column lights up, which conveys nothing, because they all matched.

An underline reads as "this column is why this row is here" without competing with marks that point at actual characters.

What I deliberately did not build

You will notice there is no @siftql/mui package here, and that is on purpose. An adapter would have to own an API for every grid library, forever, and the row and column index is application specific anyway. This one keys on row.id because that is what DataGrid guarantees. Yours might key on a composite, or on a nested path.

Roughly 40 lines in your own codebase beats a dependency that has to guess at your data shape.

Wrapping up

The load bearing idea is the first one: parse once, use twice. Everything else follows from the marks and the filter coming from the same match. Nothing is re-searched, so nothing can disagree.

• Live demo: https://siftql-demo.vercel.app/
• Query playground with 70 worked examples: https://hussein-abdallah.github.io/siftql/
• @siftql/core and @siftql/react-highlighter on npm

Both packages are MIT and have zero runtime dependencies. If you try it and something is wrong, the issue trackers are open. I would rather hear about it.