Why Homepage Email Scrapers Fail on Modern B2B Domains (And How RFC 9116 + Schema.org JSON-LD Fixes It)
When vendor risk teams, compliance workflows, or B2B revenue pipelines need to reach a company's security, privacy, billing, or legal desk, the most common automation mistake is scraping / with /[a-z0-9._%+-]+@[a-z0-9.-]
When vendor risk teams, compliance workflows, or B2B revenue pipelines need to reach a company's security, privacy, billing, or legal desk, the most common automation mistake is scraping / with /[a-z0-9._%+-]+@[a-z0-9.-]+/gi.
On modern enterprise domains, that regex fails in two predictable ways:
- It returns zero contacts because the
/homepage uses web forms or Cloudflare Email Obfuscation. - It returns asset false positives like
[email protected],[email protected], or frontend Sentry DSN strings.
While building RevClamp ContactPulse (@revclamp/b2b-domain-contact-enricher-mcp - an independent B2B domain contact and deliverability tool by RevClamp), we replaced naive homepage scraping with multi-surface protocol resolution.
1. Where Authoritative B2B Role Contacts Actually Live
Instead of guessing from raw homepage text, modern B2B domains publish verified role endpoints across three structured surfaces:
| Surface | Path / Location | Typical Roles Discovered | Standard |
|---|---|---|---|
| Security Disclosure Manifest |
/.well-known/security.txt & /security.txt
|
SECURITY, PRIVACY_DPO, LEGAL + Policy / Expires
|
RFC 9116 |
| Linked Data Graphs | <script type="application/ld+json"> |
SUPPORT, SALES, BILLING_FINANCE, PRESS
|
Schema.org ContactPoint |
| Policy & Contact Subpages |
/security, /privacy, /legal, /contact
|
Obfuscated data-cfemail & [at] / [dot] role inboxes |
HTML + Deobfuscation |
2. Parsing RFC 9116 security.txt and Schema.org ContactPoint in Node.js
RFC 9116 defines a strict line-oriented format for /.well-known/security.txt. Our parser validates Contact:, Expires:, Encryption:, Acknowledgments:, Preferred-Languages:, and Canonical: directives while checking whether Expires is in the past:
export function parseRfc9116SecurityTxt(rawBody, nowMs = Date.now()) {
const lines = String(rawBody || '').split(/\r?\n/);
const contacts = [];
let expiresIso = null;
for (const rawLine of lines) {
const line = rawLine.replace(/#.*$/, '').trim();
if (!line) continue;
const [field, ...rest] = line.split(':');
const value = rest.join(':').trim();
if (/^contact$/i.test(field) && value) contacts.push(value);
if (/^expires$/i.test(field) && value) expiresIso = value;
}
const expiresMs = expiresIso ? Date.parse(expiresIso) : NaN;
const isExpired = Number.isFinite(expiresMs) ? expiresMs < nowMs : false;
return { contacts, expiresIso, isExpired };
}
Combined with recursive Schema.org @graph and contactPoint extraction, ContactPulse classifies every discovered address into 8 canonical B2B roles (SECURITY, PRIVACY_DPO, BILLING_FINANCE, LEGAL, SUPPORT, SALES, PRESS, GENERAL) with deterministic confidence scoring.
- Companion MCP Hub & Architecture Docs: RevClamp ContactPulse on revclamp.com/mcp
- Live B2B Benchmark Hub: revclamp.com/pulse
- Apify Store Actor (
PAY_PER_EVENT): revclamp/b2b-domain-contact-enricher-mcp - Run locally via MCP
stdio:npx -y @revclamp/b2b-domain-contact-enricher-mcp
Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.