Dev.to WebDev πŸ›  Dev πŸ‘ 0 πŸ“– 5 min read

125 FAQ questions on our site, 125 distinct strings, and two pages that ask the same four questions

Notifio's site is 38 pages about a desktop app that watches rental search pages. Most of those pages end in a question block, and the questions are typed data rather than markup, so the visible accordion and the FAQPage

Notifio's site is 38 pages about a desktop app that watches rental search pages. Most of those pages end in a question block, and the questions are typed data rather than markup, so the visible accordion and the FAQPage structured data are built from the same array.

I pulled the JSON-LD out of all 38 live pages and counted.

The census

for block in re.findall(r'<script type="application/ld\+json"[^>]*>(.*?)</script>', h, re.S):
    d = json.loads(html.unescape(block))
    nodes = d.get("@graph", [d])
    for n in nodes:
        if n.get("@type") == "FAQPage":
            for e in n["mainEntity"]:
                questions.append((page, e["name"]))
  • 125 questions, on 30 of 38 pages
  • 125 distinct question strings. Zero exact duplicates
  • 4,615 words of answers, mean 36.9 words, shortest 16, longest 86
  • rendered as 4 <details> and 4 <summary> elements per site page, so the accordion and the markup cannot disagree

The distribution is almost completely flat: 28 pages carry exactly four questions. The type that governs the fifteen portal pages asks for "three or four, all specific to this site" and got four, fifteen times out of fifteen.

While I was in there, the node census for the whole site:

@type Pages carrying it
Organization 38
SoftwareApplication 33
BreadcrumbList 31
FAQPage 30
Article 8
ItemList 3
AboutPage 1

The eight pages with no FAQ are /about, /privacy, /terms, /download, /pricing and the three hub pages /alerts, /compare, /guides. A legal document, a list, and a page with one button on it are three things that should not be pretending to answer questions, so that one I am happy with.

And 125 unique strings out of 125 is exactly the result I wanted. It is also close to meaningless, which took me about a minute to work out.

The two pages that are not flat

Two pages break the four-question pattern. The home page has seven. /help has six. Put their question lists side by side:

Home /help
Which rental sites does Notifio work with? What sites does Notifio work with?
Do I need to keep the app open? Do I need to keep Notifio open on screen?
How quickly does Notifio alert me? How often does Notifio check for new listings?
Does it work on Mac and Windows? Does Notifio work on Windows and Mac?
What if a site I use does not work? Can I monitor multiple sites at once?
What if a site requires me to log in to see listings? I lost my activation token. What do I do?
Is there a subscription fee?

Four of the home page's seven questions and four of the help page's six are the same question. "Does it work on Mac and Windows?" and "Does Notifio work on Windows and Mac?" are the same eight words in a different order, and my duplicate check counted them as two distinct strings, because they are two distinct strings.

So the honest version of the headline number is this: the site has 125 question strings, 121 questions, and one pair of pages carrying a parallel FAQ that nobody is maintaining as a pair.

Why that matters more than it sounds

Because of what is inside the overlap.

Both of those pages answer "how often does it check". The home page says the app polls every 30 seconds. The help page says "by default every 60 seconds per monitor. You can adjust the interval in the monitor settings." I found and wrote about that one separately in 33 of our 38 pages say we check every 30 seconds, and three of them explain how to change a setting that does not exist, because the second half of that answer describes a settings screen the app has never had.

Both of those pages answer "how many searches can I run". The help page says "you can add as many monitors as you like". The app stops at fifteen, which I wrote about in our app refuses the sixteenth search, and 25 of our 38 pages promise there is no limit.

I found both of those by grepping for a number. Neither audit was looking for duplication, and the duplication audit I did run would never have found either, because the questions are worded differently every time. Two of the four overlapping pairs on the site's two most important pages are also the site's two worst factual contradictions, and that is not a coincidence: a question answered in two places by two edits a year apart is precisely how a product gets two answers.

The method lesson, which I have now learned twice

I did this to myself once before on another site, where exact matching reported 2 duplicated sentences in 628 and reading the pages found 5 of 6 paragraphs saying the same thing: exact-match says 2 duplicated sentences in 628 across our 67 pages, and the real answer is 5 of 6 paragraphs on one page. Same shape, different surface. A string comparison measures whether two authors typed the same characters. Nobody types the same characters.

So I tried the obvious next rung up, which needs no model: strip stop words and score the token overlap of every question against every other. Jaccard on the four pairs:

0.75  mac / windows        {and, mac, windows} vs {and, mac, notifio, windows}
0.50  which sites / sites  {notifio, rental, sites, which} vs {notifio, sites}
0.40  keep open            {app, keep, open} vs {keep, notifio, open, screen}
0.22  how quickly / often  {alert, how, me, notifio, quickly}
                           vs {check, how, listings, new, notifio, often}

A threshold of 0.4 catches three of the four. The one it does not catch is the pair at 0.22, and that is the pair carrying the 30-seconds-against-60-seconds contradiction. "Quickly" and "often" are the same question and share no characters, so the cheap trick fails exactly where the stakes are highest. This is the second time I have gone looking for duplicate copy with a lexical tool and the second time the real answer was out of its reach.

So I am not going to pretend a similarity test is the fix. The fix is upstream of the words: give each answerable question an id in the data, let more than one page claim the same id, and make the test assert that pages sharing an id give consistent answers. That turns "two pages happen to have drifted apart" into a declared relationship, and it is the only version of this check that can see a pair whose wording has nothing in common.

What I want is not deduplication. The home page should answer "do I need to keep it open", because that is the first thing a buyer asks about a monitoring app, and so should the help page. What I want is to be told that these two pages have agreed to answer the same question, so that when the answer changes it is obvious there are two of them.

You can read both lists against each other at notifio.app and notifio.app/help, and the 15 portal pages whose four questions each are the flat part of the distribution are at notifio.app/alerts.

πŸ“° Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.