Dev.to Security 🔐 Cybersecurity 👁 0 📖 2 min read

Why Homepage Email Scrapers Fail on Modern B2B Domains (And How RFC 9116 + Schema.org JSON-LD Fixes It)

When vendor risk teams, compliance workflows, or B2B revenue pipelines need to reach a company's security, privacy, billing, or legal desk, the most common automation mistake is scraping / with /[a-z0-9._%+-]+@[a-z0-9.-]

When vendor risk teams, compliance workflows, or B2B revenue pipelines need to reach a company's security, privacy, billing, or legal desk, the most common automation mistake is scraping / with /[a-z0-9._%+-]+@[a-z0-9.-]+/gi.

On modern enterprise domains, that regex fails in two predictable ways:

  1. It returns zero contacts because the / homepage uses web forms or Cloudflare Email Obfuscation.
  2. It returns asset false positives like [email protected], [email protected], or frontend Sentry DSN strings.

While building RevClamp ContactPulse (@revclamp/b2b-domain-contact-enricher-mcp - an independent B2B domain contact and deliverability tool by RevClamp), we replaced naive homepage scraping with multi-surface protocol resolution.

1. Where Authoritative B2B Role Contacts Actually Live

Instead of guessing from raw homepage text, modern B2B domains publish verified role endpoints across three structured surfaces:

Surface Path / Location Typical Roles Discovered Standard
Security Disclosure Manifest /.well-known/security.txt & /security.txt SECURITY, PRIVACY_DPO, LEGAL + Policy / Expires RFC 9116
Linked Data Graphs <script type="application/ld+json"> SUPPORT, SALES, BILLING_FINANCE, PRESS Schema.org ContactPoint
Policy & Contact Subpages /security, /privacy, /legal, /contact Obfuscated data-cfemail & [at] / [dot] role inboxes HTML + Deobfuscation

2. Parsing RFC 9116 security.txt and Schema.org ContactPoint in Node.js

RFC 9116 defines a strict line-oriented format for /.well-known/security.txt. Our parser validates Contact:, Expires:, Encryption:, Acknowledgments:, Preferred-Languages:, and Canonical: directives while checking whether Expires is in the past:

export function parseRfc9116SecurityTxt(rawBody, nowMs = Date.now()) {
  const lines = String(rawBody || '').split(/\r?\n/);
  const contacts = [];
  let expiresIso = null;
  for (const rawLine of lines) {
    const line = rawLine.replace(/#.*$/, '').trim();
    if (!line) continue;
    const [field, ...rest] = line.split(':');
    const value = rest.join(':').trim();
    if (/^contact$/i.test(field) && value) contacts.push(value);
    if (/^expires$/i.test(field) && value) expiresIso = value;
  }
  const expiresMs = expiresIso ? Date.parse(expiresIso) : NaN;
  const isExpired = Number.isFinite(expiresMs) ? expiresMs < nowMs : false;
  return { contacts, expiresIso, isExpired };
}

Combined with recursive Schema.org @graph and contactPoint extraction, ContactPulse classifies every discovered address into 8 canonical B2B roles (SECURITY, PRIVACY_DPO, BILLING_FINANCE, LEGAL, SUPPORT, SALES, PRESS, GENERAL) with deterministic confidence scoring.

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.