The 30-Minute Experiment Loop: Run AI Experiments Without a Lab Coat

Most AI advice for small businesses comes in two flavors: "try it on your real work" — which is unmeasurable — or "set up a formal A/B test" — which needs traffic volume and analyst time no 8-person shop has. Both skip the middle path that actually works at small scale.

A useful pattern arrived recently from an unexpected place: Ezra Mechaber, a growth PM at Figma, described an internal system his team calls Auto XP on the How I AI podcast. The interesting part for a small business is not the multi-agent machinery — it's the discipline wrapped around it. The system runs experiments the team would never have gotten around to, and it stops at one hard checkpoint: a human reviews the evidence and decides whether the experiment ships.

That middle path is worth translating.

Why small businesses don't experiment

If you run a service business, the honest reasons you don't test things are structural:

• No spare time — every hour goes to booked jobs and quotes.
• Low volume — a few jobs a week means no statistics; one busy Saturday skews everything.
• No clear measurement — you can't tell whether the new estimate template won more jobs or whether it was just a good month.

So improvements happen as one-off hunches, ship-and-forget, no verdict. The new follow-up sequence goes live, nobody ever checks whether it moved anything, and next year you're still paying for it on vibes.

The bounded loop, translated

Figma's version runs in a dedicated Slack channel: one loose request ("go find an experiment to run on our settings page — go crazy") triggers a data scientist agent to hunt drop-off, a UX writer to propose copy, and an engineer to ship a pull request. Every step writes back to the same thread, and one human reviews the whole package — data, hypothesis, proposed change, implementation — before anything goes live.

Here is the small business version, stripped to the parts that don't require any of that machinery:

1. Pick a surface you can see. Your quote follow-up cadence. Your Google Business Profile description. The estimate template itself. One page of your website. A rule of Auto XP is that it experiments on surfaces the team would otherwise never prioritize — at your size, that's everything, so pick one with a countable outcome: booked/not booked, replied/ghosted, click/no-click.

2. Define the change narrowly. Not "improve our follow-ups" — one specific, reversible change: the second follow-up moves from day 3 to day 5. The estimate PDF gets a one-line guarantee at the bottom. The booking page headline swaps from feature to outcome. If the change can't be stated in a sentence, you can't tell what caused the result.

3. Set the boundary before you start. This is the part small businesses skip, and it's the part that makes the loop honest. Before anything changes, write three lines: how long it runs (two weeks, not "a while"), what data you'll look at (reply rate on follow-ups, booked-rate on estimates sent), and what result ends it early (a customer complains, or a reply rate tanks). Figma's checkpoint is a human reviewing the whole package before launch; your version is the same judgment written down in advance, when you're not attached to the outcome.

4. One human checkpoint. In Figma's loop, agents can do almost everything — except the go/no-go call. The rule worth copying exactly: the person who decides whether the change ships is not the person (or the agent) who proposed it. In a one-owner shop, that judgment is the whole system. The agent drafts, you decide.

5. Write the verdict, then move on. Two lines in a log: what you tried, what happened. No dashboard, no spreadsheet — a running file. The compounding asset is the log: next year's "should we try X" question gets answered by what the log says, not by re-litigating from memory.

The three failure modes, and the cheap defenses

Figma's write-up notes the loop's weak points honestly, and all three shrink to small-business scale:

• Weak hypothesis from thin data. Their fix: stricter criteria for what counts as a good opportunity. Yours: never run an experiment on a surface with fewer than ~20 outcomes a month; at lower volume, the noise eats the signal and you're just reading tea leaves.
• Implementation bugs. Their fix: automated tests against the PR. Yours: a one-line reversal plan written before launch ("delete the sentence, restore the old template") — if the experiment goes wrong, you undo it in the time it takes to notice.
• The loop gets ambitious. Figma's team is running five concurrent experiments on surfaces nobody prioritized. A small business should run one at a time. Concurrency is how you lose track of what caused what — and it multiplies the review burden past one human checkpoint.

Where the agent actually helps

In the Figma version, agents compress the work between the idea and the checkpoint: analysis, drafting, implementation. At small-business scale, the agent's honest job is narrower but real:

• Draft the change. "Here's our current estimate email and our guarantee policy — draft two subject-line variants, plain, no hype."
• Build the evidence package. Paste the last two weeks of follow-up replies and ask: "Reply rate before vs. after? Count them, don't guess." A cheap fast model counts better than an owner eyeballs.
• Write the verdict line. "Given the data we agreed on, what's the two-line log entry?"

What the agent does not do: pick the surface, define the change, set the boundary, or make the go/no-go call. Those four lines are the human checkpoint, and at small scale they're the entire management layer of the loop.

Honest boundaries

The Figma team reports a "huge win" from their first experiment and five concurrent live experiments on surfaces nobody prioritized. They also report the system is still approaching its intended operating model — Mechaber's own description is that the theory is great and the practice is "almost there." No lift figures or statistical detail have been published, and the loop draws on his accumulated growth-PM instructions — it is not evidence that a generic agent can be handed experimental judgment. Take the pattern, not the hype: the durable part is the boundary-plus-checkpoint discipline, which works whether the drafting is done by an agent or by you on a Sunday afternoon.

The other boundary: not everything needs to be an experiment. If a change is cheap, obviously right, and reversible, just make it. The loop is for changes where you'd otherwise guess — pricing-adjacent copy, follow-up cadence, what your estimate leads with. Save the discipline for decisions worth a verdict.

Your first loop, tonight

1. Pick one surface with a countable outcome.
2. Write the three boundary lines: duration, metric, kill switch.
3. Have an agent (or yourself) draft the one-sentence change and the reversal plan.
4. Launch it. Touch nothing for two weeks.
5. On the checkpoint date, write the two-line verdict. Keep, revert, or extend one more cycle.

Total cost: about thirty minutes of setup and two minutes a day of not fiddling. That last part is the real skill.

Want the boring half pre-built? The Boring Automation Pack from SMB Scale Up includes plug-and-play automation blueprints plus the standing-instructions worksheet — the templates and tracking files that make loops like this run without babysitting.