Running SEO Improvements as One Flow, from Proposal to Verified Effect — AIO Helper

Conclusion

AIO Helper is an SEO operations SaaS we are building. It ingests search data, has AI propose page-level improvements, applies the proposals a person approves, and then checks whether each change actually worked. These four stages run on one screen and one database.

• Ingest. It pulls page- and keyword-level numbers every day from Google Search Console and other sources.
• Propose. AI proposes improvements per page: titles and descriptions, internal links, additional content, and more. Each proposal carries an expected impact, a confidence level, a risk, and an effort.
• Apply. Approved proposals are applied to pages. Titles and descriptions are written to AIO Helper's per-page SEO settings, which the site reads through an API. For pages on our own CMS, Plovant, there is also a setting that applies low-risk proposals automatically every day.
• Verify. It observes each applied change in 14-day windows and sorts the result into "improved," "no change," "worsened," "insufficient data," and so on, using rankings, clicks, and impressions.

The history of applied changes, and the lessons people record after looking at each verification, go back into the material the AI searches when it writes the next proposal. This article walks through the four stages with the actual screens and code.

Main text (about 9 min read)

Why I Built It

AIO Helper started as the SEO features inside our CMS, Plovant. Plovant was first built as an SEO platform. Later, a CMS and an SEO product began to live in the same repository, so I split the SEO features out into a separate product. That product is AIO Helper.

When I split it out, I asked what the most laborious part of SEO actually is.

Producing improvement ideas is not that hard anymore. Search Console numbers tell you which pages "get many impressions but few clicks," and if you ask an AI, it will suggest a rewritten title.

The hard part comes after that.

• Which of the proposed ideas actually went onto a page?
• When did it go in, and what were the rankings and clicks before the change?
• After it went in, did it work, or did it make things worse?
• Is it safe to roll a change that worked out to other pages?

Once you start tracking this in a spreadsheet, you quickly lose track. The place where ideas come from, the place where pages are rewritten, and the place where you look at the numbers are all separate.

So in AIO Helper, each improvement idea is held as a single record called a "proposal," and that record changes state as it is approved, applied, and verified. The point is to be able to trace, proposal by proposal, what was done to which page and how the metrics changed.

The AIO Helper Screens

The admin UI is split into four groups in the left menu.

CONTROL   Control Room
STRATEGY  Strategy Pipeline / Expansion Roadmap / Growth Structure
INSIGHTS  Keywords / Pages / Page Goals / Content / Site Structure /
          E-E-A-T / Links / Performance / Verification
SYSTEM    Reports / Settings

At the top of the screen, you can switch the viewer's role between "Operator," "Manager," and "Executive." The same data is shown differently for the person running daily changes, the person setting priorities, and the person who only wants the overall numbers.

From here, let's go through the four stages in order.

1. Ingest

The first stage is ingesting search data. The core is Google Search Console data; there are also ingestion jobs for Google Analytics 4, PageSpeed Insights, and ad data.

The daily jobs run in a fixed order. Times are in Japan Standard Time.

03:30  Ingest Search Console data
04:00  Re-embed the site's documents (preparation for the search described later)
04:10  Aggregate before/after numbers for applied changes
04:20  Crawl our own site and collect page state
04:45  Generate proposals in a batch
05:00  Verify the effect of applied changes
05:30  Apply low-risk proposals to page drafts

The order is set so that the numbers ingested that day feed that day's proposals and verifications.

What I had to be careful about in ingestion is that Search Console data takes time to finalize. If you fetch yesterday's data on the same day, it can still be zero rows. If you treat zero rows as "ingestion complete," that day's data stays missing.

So the daily ingestion re-fetches 7 days at a time. By default, that is the 7 days ending 3 days ago.

// seo-ingest-gsc/src/dates.ts
export const DEFAULT_REFETCH_DAYS = 7;

Days that were already ingested are overwritten with the finalized values. Fetching the same day again does not change the result, so repeating the re-fetch every day does not create duplicates. This "completed with zero rows and left a gap" problem is one I fixed after it actually happened. I will cover the details in another article in this series.

Ingested numbers go into the database per page, per keyword, and per day. Besides the rankings of tracked keywords, the Keywords screen shows "keywords that have search volume but where the site does not rank yet" as gaps.

2. Propose

The second stage is AI-generated improvement proposals. The "Strategy Pipeline" screen lists proposals per page.

A single proposal is data shaped like this. I took one demo-data item straight from the API response.

{
  "type": "Meta",
  "target": "/pricing",
  "locale": "ja",
  "impact": 82,
  "confidence": 91,
  "risk": "Low",
  "risk_reason": "Title and description change only. No structural change.",
  "effort": "Low",
  "status": "New",
  "evidence_tags": ["GSC", "SERP", "GA4"],
  "summary": "/pricing has no title or description set. Expected +120 clicks per month from better CTR"
}

(The risk_reason and summary values are Japanese in the actual response; they are translated here.)

The main proposal types include title and description changes (Meta), body rewrites (Content), adding a group of related pages (Cluster), internal links (InternalLink), site structure (Architecture), page speed (Performance), and brand notation (Brand).

Each comes with "impact," "confidence," "risk," and "effort." The screen can sort by impact, confidence, and risk, and filter by type, risk, and effort, so you can work through "high impact, low risk" items first. Risk also carries a reason, because rewriting a title and adding three new pages have completely different consequences when they fail.

Using Past Verification Results as Evidence for Proposals

When the AI writes a proposal, it does not start from nothing. It first pulls documents about the site with vector search (a way of finding text by closeness of meaning), and then writes the proposal. The ordering of that search is the following code.

SELECT d.id AS doc_id, d.doc_type, d.title, d.content,
       (e.embedding <=> $1::vector) AS distance
  FROM seo_embeddings e
  JOIN seo_docs d ON d.id = e.doc_id
 WHERE e.embedding IS NOT NULL
 ORDER BY
   CASE WHEN d.site_id = $2 THEN 0 ELSE 1 END,
   CASE d.doc_type
     WHEN 'brand_rules'    THEN 0
     WHEN 'learning'       THEN 1
     WHEN 'outcome_report' THEN 2
     WHEN 'change_log'     THEN 3
     ELSE 4
   END,
   e.embedding <=> $1::vector
 LIMIT 8

The ordering means this:

1. First, the site's own documents take priority.
2. Within those, brand notation rules (brand_rules) come first. A proposal that gets the company or product name wrong is unusable however much impact it promises.
3. Next comes learning. These are records a person attaches to a proposal after looking at its result on the verification screen: "which type of proposal went onto which page, what the result was," and what was learned from it, one by one.
4. After that come outcome reports (outcome_report) and the change history (change_log). The change_log is recorded automatically every time a page's SEO settings change.

In other words, changes that worked before and changes that did not are pulled near the top as evidence when the next proposal is written. This is one reason not to stop at proposing, but to follow through to verifying the effect. Without verification records, the AI would produce similar generic proposals every time.

3. Apply

The third stage is applying changes to pages.

There are two paths.

The first writes to AIO Helper's per-page SEO settings. Changing a title or description updates these settings (PUT /v1/seo). The site reads these SEO settings from the API (/v1/seo) and uses them when rendering the page. There is also a small library for that.

// packages/seo-kit/src/fetch-seo.ts (excerpt)
const url = `${SEO_API_URL}/v1/seo?siteId=${encodeURIComponent(siteId)}&path=${encodeURIComponent(path)}&locale=${encodeURIComponent(locale)}`;
const res = await fetch(url, { cache: "no-store" });

Along with the write, it does three things:

• Saves the before and after as a change_log, which becomes search material for the next proposal
• Sends seo.updated to registered endpoints as a signed notification (webhook)
• Resubmits the sitemap to Search Console

The second path applies changes directly to pages on our CMS, Plovant. When enabled in a site's settings, low-risk proposals are applied to the Plovant page drafts automatically every day. Whether to go further and publish those drafts automatically is a separate setting.

I split automatic application into two levels, "up to the draft" and "up to publishing," because if AI proposals are published as-is and rankings drop, rolling back is painful. The first step is a state where the content can be checked in a draft, and each site decides whether to hand over publishing as well.

4. Verify

The fourth stage is verifying the effect. What gets verified are changes registered as a pair of a page and its target keyword; not every applied proposal becomes a verification target automatically. The "Verification" screen lists the result for each registered change.

Verification splits the time after publishing into 14-day windows and compares each with the period before the change, looking at how the target keyword's ranking and the page's clicks moved. Observation normally runs until 56 days after publishing. The thresholds are:

// rank-watch-judge.ts
export const DEFAULT_JUDGE_CONFIG: JudgeConfig = {
  minImpressions: 50,        // below this many impressions in the period, draw no conclusion
  improvedDelta: 0.5,        // ranking better by at least this: "improved"
  worsenedRankDelta: 2.0,    // ranking worse by at least this: "worsened"
  clickGuardRatio: 0.85,     // page clicks below 85% of before: "worsened"
  clickGuardMinBaseline: 10, // fewer than 10 clicks before: don't use clicks
  imprGuardRatio: 0.7,       // page impressions below 70% of before: "worsened"
  imprGuardMinBaseline: 100, // fewer than 100 impressions before: don't use impressions
};

What I cared about most here is keeping "insufficient data" as a result of verification.

Search numbers fluctuate to begin with. If a keyword gets only a handful of impressions and its ranking rises by one, you cannot tell whether that is the effect of the change or chance. So if the target keyword does not reach 50 impressions within the period, it says neither "improved" nor "worsened" and only records that the data is insufficient.

Also, even if the target keyword's ranking goes up, the result is "worsened" if the page's total clicks drop sharply. This avoids looking only at one keyword and missing that the page as a whole is losing.

A person who looks at the result can attach a learning record to that proposal. As described in the previous section, this record is searched as evidence for the next proposal. Changes that came out "worsened" appear under "Next actions" on the screen as items to consider rolling back.

Limits of This Design

AIO Helper's design has clear limits.

• Pages with few impressions often get no conclusion. The numbers needed for verification don't come together, so new pages and pages with little search traffic stay at "insufficient data." Observation can end as "insufficient data" before the data comes together. Checking that change again later requires registering it again.
• Ranking changes cannot be attributed to the change alone. Rankings also move because of search-engine updates or changes in competing pages. Because the method compares before and after, other changes that happened in the same period cannot be separated out.
• AI costs money. Every proposal calls a generative AI API and a vector-search API, so cost grows with use. That is why each site has a monthly cap, and requests whose estimated cost exceeds the remaining budget are stopped before the AI is called. Because the check uses an estimate, it cannot prevent overruns when the actual cost exceeds the estimate or when concurrent requests arrive together. I will explain this mechanism in detail in the next article.

What This Series Will Cover

The AIO Helper articles start from this introduction and go on to explain each feature and the decisions made while building it. Planned topics:

• Stopping AI spend inside the app before the invoice arrives
• Search Console's delayed finalization, and yesterday's data coming back as zero rows
• Moving a rarely called API from an always-on server to a setup that runs only when needed
• Viewing the language versions of the same page as one, on a multilingual site
• The AI no longer producing proposals, and preventing it in both the prompt and the database
• Why vector search should be tested against real Postgres

Summary

• The laborious part of SEO improvement is not producing ideas but continuing to track what went onto which page and what happened afterward.
• AIO Helper holds each improvement idea as a single "proposal" record and moves it through four stages: ingest, propose, apply, and verify.
• Verification keeps "insufficient data" as a result too, and says neither "improved" nor "worsened" when impressions are insufficient.
• The change history and the lessons recorded from verification results go back into the search material for the next proposal, so the improvement record itself becomes the evidence for the next improvement.