Dev.to WebDev πŸ›  Dev πŸ‘ 0 πŸ“– 4 min read

Replacing Clearbit & Expensive Enrichment APIs: A 128MB Pure-HTTP Company-to-Domain, LinkedIn & Email Resolver (Fixing Cheerio OOM)

Enriching a list of raw B2B leads typically forces teams to stitch together three separate APIs or SaaS subscriptions: A company-name-to-domain resolver ("Stripe" -> stripe.com) A company-to-LinkedIn URL lookup (https:

Enriching a list of raw B2B leads typically forces teams to stitch together three separate APIs or SaaS subscriptions:

  1. A company-name-to-domain resolver ("Stripe" -> stripe.com)
  2. A company-to-LinkedIn URL lookup (https://www.linkedin.com/company/stripe)
  3. A contact scraper (sales@, hello@, WhatsApp links, international phones, socials)

Paying $8/1K + $4/1K + $5/1K across fragmented tools quickly wrecks unit economics. Worse, running headless browsers across 2,000 corporate homepages creates memory bloat and billing surprises.

Here is how I engineered a 3-in-1 pure-HTTP B2B company-to-domain, LinkedIn, email, and WhatsApp enricher that runs comfortably inside a 128 MB RAM container β€” including the exact fix for a nasty Cheerio OOM crash on 600KB–2MB enterprise homepages.

1. The 128 MB Container Killer: cheerio.load() on Giant Modern Homepages

Modern SaaS homepages (like stripe.com, linear.app, or enterprise Webflow/Next.js sites) routinely ship 600 KB to 2 MB of raw HTML β€” most of it stuffed inside inline <script> hydration bundles, <style> CSS-in-JS blocks, and hundreds of inline <svg> icons.

When you pass 1.5 MB of raw HTML directly into cheerio.load(html) across 5–6 concurrent workers, parse trees can spike ~70 MB per page, crashing a 128 MB container with ENOMEM.

The Pre-Cheerio slimHtml Fix (~70 MB -> ~3 MB RAM)

Because our enricher needs structured metadata (application/ld+json Organization schemas), visible links, and contact text β€” not megabytes of CSS and inline SVG paths β€” we extract JSON-LD blocks with a quick regex and strip <script>, <style>, <svg>, <noscript>, and <template> before invoking cheerio.load():

const rawJsonLdBlocks = [];
for (const m of html.matchAll(/<script\b[^>]*type=["']application\/ld\+json["'][^>]*>([\s\S]*?)<\/script>/gi)) {
    if (m[1]) rawJsonLdBlocks.push(m[1]);
}

const slimHtml = html
    .replace(/<script\b[\s\S]*?<\/script>/gi, ' ')
    .replace(/<style\b[\s\S]*?<\/style>/gi, ' ')
    .replace(/<svg\b[\s\S]*?<\/svg>/gi, ' ')
    .replace(/<noscript\b[\s\S]*?<\/noscript>/gi, ' ')
    .replace(/<template\b[\s\S]*?<\/template>/gi, ' ');

const $ = cheerio.load(slimHtml);

Along with a strict 2 MB streaming byte cap, binary MIME skipping (PDFs, images, ZIPs), and sequential /contact + /about subpage fetching, memory stays flat on 128 MB containers.

2. Clean Email Prioritization & LinkedIn Strict Filtering

Scraping emails blindly produces garbage like [email protected], [email protected], [email protected], and [email protected]. Similarly, naive regexes on homepages frequently match a founder's personal /in/... profile or a linkedin.com/sharing/share-offsite button instead of the company page.

Two strict normalizers eliminate 99% of that noise:

Rejecting Asset Filenames & Prioritizing Role Emails on Company Domains

cleanEmail() strips mailto:, drops excluded extensions (.png, .svg, .webp, .woff2), blocks retina [email protected] prefixes (^\d+x\.), and filters noreply@ / SaaS error-tracker domains. Then sortEmails() ranks role emails (sales@, contact@, hello@, info@, team@) matching the verified corporate domain above third-party addresses.

Strict Company-Only LinkedIn Normalization

export function normalizeLinkedinCompanyUrl(rawUrl) {
    if (!rawUrl || typeof rawUrl !== 'string') return null;
    try {
        const candidate = rawUrl.startsWith('//') ? `https:${rawUrl}` : rawUrl.trim();
        const u = new URL(candidate);
        const host = u.hostname.toLowerCase();
        if (!(host === 'linkedin.com' || host.endsWith('.linkedin.com'))) return null;

        const path = u.pathname.replace(/\/+$/, '');
        if (/^\/(in|pub|sharing|shareArticle|feed|jobs|posts|pulse|learning|events)\b/i.test(path)) {
            return null;
        }
        const match = path.match(/^\/(company|school)\/([a-zA-Z0-9._%-]+)/i);
        if (!match) return null;
        const slug = match[2].replace(/\/.*$/, '');
        if (!slug || slug.toLowerCase() === 'unavailable') return null;
        return `https://www.linkedin.com/${match[1].toLowerCase()}/${slug}`;
    } catch {
        return null;
    }
}

3. 3-in-1 Resolution Without Search-Engine Scraping

Given any mix of company names ("Linear", "Canva Pty Ltd"), domains (canva.com), work emails ([email protected]), or an upstream Apify datasetId (such as Google Maps or Apollo results):

  1. Query public structured Wikidata identifiers + deterministic legal-suffix stripping & TLD candidate probing with live DNS verification.
  2. Fetch the verified homepage and score brand-match confidence (pageTitle, og:site_name, Schema.org Organization, copyright notices) while rejecting parked/for-sale domains (sedo, dan.com, hugedomains).
  3. If emails, WhatsApp (wa.me / api.whatsapp.com), or LinkedIn company links are missing on the homepage, sequentially crawl up to 2 internal /contact or /about pages.

Misses, parked domains, free webmails (gmail.com), and SSRF-blocked inputs are never billed ($0).

Try It Live or Chain With Google Sheets / n8n / MCP

Import 03-b2b-company-linkedin-email-enricher-sheets.json into n8n to turn a raw Input tab of company names into an enriched CRM table with verified domains, LinkedIn URLs, role emails, and WhatsApp links in a single run.

πŸ“° Read the original article on Dev.to WebDev

Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes β€” full credit and traffic to the original publisher.