Scoping an AI automation pilot: start with one workflow

By JC TechLabs

An AI automation pilot is easy to start and surprisingly hard to finish well. The demo works in an afternoon. Then the questions arrive. What exactly did it improve? What happens when the input is wrong? Who runs it next month?

Most of those problems trace back to scope. Pilots that try to cover a department rarely produce a clear answer. Pilots that cover one workflow usually do. Here's how we approach scoping one.

Examples below are reference scenarios for planning and do not report JC TechLabs client results.

1. Choose one workflow, not a department

A good first candidate has four properties:

• It repeats. It happens often enough that patterns are visible.
• It has edges. A clear trigger (an enquiry arrives, a form is submitted) and a clear finished state.
• It has an owner. Someone who can say "this output is right" and who will run it afterwards.
• Mistakes are recoverable. Early pilots belong where a wrong draft can be caught, not where it silently reaches a customer or a ledger.

"Answer first-line product questions from the published help content" is a workflow. "Automate customer service" is a programme.

If you're not sure where to start, a planning pass like our Automation Planner helps you list candidate tasks and compare them before anyone writes code.

2. Record the baseline before you change anything

Without a baseline, every pilot looks like a success. Before building, record how the workflow runs today:

• handling time per item (a sample is fine)
• quality: how often the output needs correction
• handoff rate: how often a person has to step in
• volume and its peaks

Then evaluate the pilot against that baseline before extending it. And be careful with the arithmetic: released staff capacity is not automatically a cash saving. Hours moved into a review queue are still hours.

3. Draw the AI automation pilot boundary

Write the boundary down as a short document everyone signs off:

IN SCOPE    : first-line questions covered by published help articles
OUT OF SCOPE: refunds, complaints, account changes, anything legal
MUST HAND OFF: low confidence, missing information, angry or sensitive tone
NEVER ALONE : sending money, changing records, promising outcomes
DATA        : which systems it may read, which it may write (least privilege)

The "never alone" line matters most. It tells the team where human-in-the-loop review is not optional.

4. Design human review as part of the workflow

Human review shouldn't be a safety net bolted on afterwards. Decide up front:

• what goes to review: low-confidence outputs, out-of-scope requests, anything in the "never alone" list
• who reviews it, and how quickly
• what the reviewer sees: the original input, the proposed output and the reason it was flagged
• how corrections feed back into prompts, rules or evaluation cases

Count the review queue as part of the workflow's cost. It's real work.

5. Test the unhappy paths

A pilot that only proves the happy path proves very little. Build evaluation cases for:

• incomplete inputs
• unsupported questions
• duplicate submissions
• an unavailable integration or a slow model

For each one, define the expected behaviour. Usually that means a clean handoff, a retry, or a clear message, rather than a confident guess. Our notes on engineering resilient business systems cover retries and recovery paths in more depth.

6. Agree acceptance criteria and a stop rule

Before the pilot starts, agree what "good enough to extend" means, in terms of quality, handoff rate and operating cost per task against the baseline. Also agree what ends the pilot early. A stop rule protects everyone from sunk-cost momentum.

7. Plan the handover on day one

The pilot should end with something a team can run:

• a workflow map and the signed boundary
• evaluation cases and their latest results
• a runbook: how to pause it, who fixes it, where the logs are
• an operating-cost view of models, integrations and review time

If no one can own it after handover, it isn't ready to scale.

Rules or AI?

Not every workflow needs a model. Stable, rule-shaped tasks are often better served by deterministic automation. We compare the two in RPA rules vs AI workflows, and our AI automation guide covers examples and planning.

The short version

One workflow. A recorded baseline. A written boundary. Human review where it matters. Tested failure paths. A clear stop rule and a real handover.

That's an AI automation pilot that produces an answer, whatever the answer turns out to be.

JC TechLabs plans and builds AI automation, including bounded pilots, human-review workflows and rollout planning. If you have a workflow in mind, **discuss your project: AI & Automation · Start a conversation. A public service description is not a signed delivery commitment. Scope, milestones and commercial terms are agreed before work begins.