Building an Enterprise-Grade Web Metadata & Email Extractor API (

Hello DEV Community! 👋

I want to share a project I've been building: an ultra-fast Web Metadata, OpenGraph & Contact Extractor API.

Here is the interesting part: 100% of the code, architecture, optimizations, and benchmarks were engineered in partnership with Gemini AI (Antigravity).

Initially, heavy web scraping took up to 23+ seconds on bloated websites. Working together with Gemini AI, we refactored and optimized the entire backend step-by-step to achieve enterprise-grade performance under 200ms (and 0.05ms on cache hits!).

🛠️ The Technical Stack & Optimizations

Here is how Gemini AI brought response times down by 99%:

1. Fast C-Parser (selectolax): Replaced standard BeautifulSoup parsing with selectolax (written in C based on Lexbor engine), dropping DOM parsing times from 15ms to ~2ms.
2. HTTP/2 & 256KB Streaming Limit: Configured httpx with native HTTP/2 multiplexing and stream capping at 256KB to retrieve <head> tags instantly without downloading 20MB of unnecessary assets.
3. Smart DOM Cleaning: Stripped <script>, <style>, and <code> tags before running regexes to prevent email false positives (like [email protected] in JS templates).
4. 15-Min TTL In-Memory Cache: Repeated requests to identical URLs return instantaneously in 0.05 ms.

🔍 What the API Extracts in a Single Call:

• 🎯 SEO & OpenGraph Metadata: Title, Description, OG Image, Keywords, Author, Language, Favicon.
• 📧 Clean Contact Discovery: Public emails & phone numbers (cleaned of JS noise).
• 📲 Social Media Profiles: Auto-detects Twitter/X, LinkedIn, Facebook, Instagram, GitHub, YouTube, Telegram, and TikTok.
• 🛠️ Tech Stack Detector: Identifies WordPress, Shopify, WooCommerce, React, Next.js, Vue, Nuxt, TailwindCSS, Bootstrap.

🎁 Free Tier Available for Developers

I opened a free tier of 100 requests/month on RapidAPI so the community can use it for side-projects, lead enrichment, or link preview cards:

• 📦 GitHub Repository & Code Examples (Python, JS, cURL): https://github.com/JosejuX/rapidapi-metadata-extractor
• 🔑 RapidAPI Hub: Web Metadata & Contact Extractor on RapidAPI

💬 Final Thoughts & Feedback

Working with Gemini AI to build this API showed me how fast we can take an idea from zero to a production-ready, monetization-capable SaaS.

I’d love to get your feedback, suggestions, or feature requests! What do you think about AI-driven API development? Let me know in the comments below!