How I Designed a Website QA Pipeline That Combines Browser Automation, Deterministic Checks, and AI

How I Designed a Website QA Pipeline That Combines Browser Automation, Deterministic Checks, and AI

Building a website auditing system sounds straightforward at first:

Enter a URL → crawl the website → find problems → generate a report.

In practice, a reliable website QA system is much more complicated.

A useful auditor needs to understand the actual rendered website, inspect technical behaviour, detect SEO problems, evaluate accessibility, measure performance, identify security issues, and produce results that developers can actually act on.

AI can help with some of these tasks, but relying on AI for everything creates another set of problems: inconsistent results, hallucinations, unpredictable scoring, and unnecessary API costs.

For one of my recent engineering projects, I designed the QA pipeline around a simple principle:

Use deterministic systems to measure. Use AI to explain and assist.

This article explains the architecture and engineering decisions behind that approach.

-> The Core Problem

A website audit normally contains two very different types of work.

The first type is objective:

• Does a page return HTTP 200?
• Is a link broken?
• Does the page have an H1?
• Is a canonical URL present?
• Is HTTPS enabled?
• Are security headers configured?
• What is the page's TTFB?
• Are images missing alternative text?

These are problems where software should be able to produce the same answer repeatedly.

The second type is interpretive:

• Why is this issue important?
• What should the developer change?
• How should the problem be fixed?
• Which issues deserve attention first?
• How should the findings be explained to a client?

This is where AI becomes useful.

Instead of asking an LLM to perform the entire audit, I separated the system into multiple layers.

Website
↓
Crawler
↓
Browser Automation
↓
Normalization
↓
Deterministic QA Engine
↓
Scoring + Findings
↓
AI Analysis
↓
Fix Suggestions + Reports

The main lesson I took from building this system is that AI is most powerful when it is given a well-defined job.

It doesn’t have to discover everything.

It doesn’t have to make every decision.

And it definitely shouldn’t be trusted blindly to modify production systems.

Traditional software is extremely good at measuring things that can be defined precisely.

AI is extremely useful when the problem requires interpretation, explanation, prioritization, or generating contextual recommendations.

Combining both approaches creates a much stronger system than trying to make either one do everything.

Measure with software.

Explain with AI.

Fix carefully.

Verify with software again.

That is the architecture I would choose when building a modern website auditing and QA platform intended for real development and agency workflows.

PROJECT

I built this approach into ScanFix, a Website QA & Audit platform designed around automated website analysis, deterministic scoring, AI-assisted analysis, and professional reporting.

You can learn more about the project here:

https://uniteelements.com/offer-os/scanfix/

This article was prepared with assistance from AI for drafting and language refinement. I reviewed the technical concepts and structure before publication, and the implementation details described here are based on my own project and engineering work.