Replacing Clearbit & Expensive Enrichment APIs: A 128MB Pure-HTTP Company-to-Domain, LinkedIn & Email Resolver (Fixing Cheerio OOM)
Enriching a list of raw B2B leads typically forces teams to stitch together three separate APIs or SaaS subscriptions: A company-name-to-domain resolver ("Stripe" -> stripe.com) A company-to-LinkedIn URL lookup (https:
Enriching a list of raw B2B leads typically forces teams to stitch together three separate APIs or SaaS subscriptions:
- A company-name-to-domain resolver (
"Stripe"->stripe.com) - A company-to-LinkedIn URL lookup (
https://www.linkedin.com/company/stripe) - A contact scraper (
sales@,hello@, WhatsApp links, international phones, socials)
Paying $8/1K + $4/1K + $5/1K across fragmented tools quickly wrecks unit economics. Worse, running headless browsers across 2,000 corporate homepages creates memory bloat and billing surprises.
Here is how I engineered a 3-in-1 pure-HTTP B2B company-to-domain, LinkedIn, email, and WhatsApp enricher that runs comfortably inside a 128 MB RAM container β including the exact fix for a nasty Cheerio OOM crash on 600KBβ2MB enterprise homepages.
1. The 128 MB Container Killer: cheerio.load() on Giant Modern Homepages
Modern SaaS homepages (like stripe.com, linear.app, or enterprise Webflow/Next.js sites) routinely ship 600 KB to 2 MB of raw HTML β most of it stuffed inside inline <script> hydration bundles, <style> CSS-in-JS blocks, and hundreds of inline <svg> icons.
When you pass 1.5 MB of raw HTML directly into cheerio.load(html) across 5β6 concurrent workers, parse trees can spike ~70 MB per page, crashing a 128 MB container with ENOMEM.
The Pre-Cheerio slimHtml Fix (~70 MB -> ~3 MB RAM)
Because our enricher needs structured metadata (application/ld+json Organization schemas), visible links, and contact text β not megabytes of CSS and inline SVG paths β we extract JSON-LD blocks with a quick regex and strip <script>, <style>, <svg>, <noscript>, and <template> before invoking cheerio.load():
const rawJsonLdBlocks = [];
for (const m of html.matchAll(/<script\b[^>]*type=["']application\/ld\+json["'][^>]*>([\s\S]*?)<\/script>/gi)) {
if (m[1]) rawJsonLdBlocks.push(m[1]);
}
const slimHtml = html
.replace(/<script\b[\s\S]*?<\/script>/gi, ' ')
.replace(/<style\b[\s\S]*?<\/style>/gi, ' ')
.replace(/<svg\b[\s\S]*?<\/svg>/gi, ' ')
.replace(/<noscript\b[\s\S]*?<\/noscript>/gi, ' ')
.replace(/<template\b[\s\S]*?<\/template>/gi, ' ');
const $ = cheerio.load(slimHtml);
Along with a strict 2 MB streaming byte cap, binary MIME skipping (PDFs, images, ZIPs), and sequential /contact + /about subpage fetching, memory stays flat on 128 MB containers.
2. Clean Email Prioritization & LinkedIn Strict Filtering
Scraping emails blindly produces garbage like [email protected], [email protected], [email protected], and [email protected]. Similarly, naive regexes on homepages frequently match a founder's personal /in/... profile or a linkedin.com/sharing/share-offsite button instead of the company page.
Two strict normalizers eliminate 99% of that noise:
Rejecting Asset Filenames & Prioritizing Role Emails on Company Domains
cleanEmail() strips mailto:, drops excluded extensions (.png, .svg, .webp, .woff2), blocks retina [email protected] prefixes (^\d+x\.), and filters noreply@ / SaaS error-tracker domains. Then sortEmails() ranks role emails (sales@, contact@, hello@, info@, team@) matching the verified corporate domain above third-party addresses.
Strict Company-Only LinkedIn Normalization
export function normalizeLinkedinCompanyUrl(rawUrl) {
if (!rawUrl || typeof rawUrl !== 'string') return null;
try {
const candidate = rawUrl.startsWith('//') ? `https:${rawUrl}` : rawUrl.trim();
const u = new URL(candidate);
const host = u.hostname.toLowerCase();
if (!(host === 'linkedin.com' || host.endsWith('.linkedin.com'))) return null;
const path = u.pathname.replace(/\/+$/, '');
if (/^\/(in|pub|sharing|shareArticle|feed|jobs|posts|pulse|learning|events)\b/i.test(path)) {
return null;
}
const match = path.match(/^\/(company|school)\/([a-zA-Z0-9._%-]+)/i);
if (!match) return null;
const slug = match[2].replace(/\/.*$/, '');
if (!slug || slug.toLowerCase() === 'unavailable') return null;
return `https://www.linkedin.com/${match[1].toLowerCase()}/${slug}`;
} catch {
return null;
}
}
3. 3-in-1 Resolution Without Search-Engine Scraping
Given any mix of company names ("Linear", "Canva Pty Ltd"), domains (canva.com), work emails ([email protected]), or an upstream Apify datasetId (such as Google Maps or Apollo results):
- Query public structured Wikidata identifiers + deterministic legal-suffix stripping & TLD candidate probing with live DNS verification.
- Fetch the verified homepage and score brand-match confidence (
pageTitle,og:site_name, Schema.orgOrganization, copyright notices) while rejecting parked/for-sale domains (sedo,dan.com,hugedomains). - If emails, WhatsApp (
wa.me/api.whatsapp.com), or LinkedIn company links are missing on the homepage, sequentially crawl up to 2 internal/contactor/aboutpages.
Misses, parked domains, free webmails (gmail.com), and SSRF-blocked inputs are never billed ($0).
Try It Live or Chain With Google Sheets / n8n / MCP
-
Apify Store Actor (
$2.50 / 1,000enriched companies,$0unresolved): B2B Company to Domain, LinkedIn, Email & WhatsApp API -
Open-Source Starter Kit (n8n Google Sheets Workflow + MCP + Node/Python): GitHub β
apify-b2b-ecom-automation-kit
Import 03-b2b-company-linkedin-email-enricher-sheets.json into n8n to turn a raw Input tab of company names into an enriched CRM table with verified domains, LinkedIn URLs, role emails, and WhatsApp links in a single run.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.