ARC-AI: I built an autonomous, AI agent workspace Using MERN stack (RAG memory, live web research, WhatsApp automation, and more)

🤖 What is ARC-AI?

ARC-AI (Autonomous Real-time Conversational Agent) is a full-stack, voice-activated AI assistant I've been building that goes beyond the typical "chatbot with a system prompt." It's a persistent, AI workspace with long-term memory, live web research, proactive scheduled routines, and the ability to actually act — sending WhatsApp messages, emails, and controlling its own UI.

Think of it as sitting somewhere between ChatGPT, Claude, and Cursor — but self-hosted, MERN-based, and fully yours.

🔗 Live demo: https://arc-ai-project.vercel.app/
🔗 Source code: https://github.com/Aashutosh31/arc-ai-project
📽️ Latest Video:https://www.instagram.com/aashutosh_bairagi.31/reel/DYmluEPxOhf/
🎥 Full walkthrough: https://www.youtube.com/watch?v=jt7q8v5KsrU

Why I built it

Most "AI assistants" are reactive — you ask, it answers, and it forgets everything the moment the tab closes. I wanted something that behaves more like an actual agent: it remembers what matters, researches things on its own, runs scheduled tasks in the background, and can reach across into the real world (browser, WhatsApp, email) instead of being boxed into a chat window.

That meant building real infrastructure, not just wiring up an LLM API call — a provider-agnostic runtime, workspace isolation, vector memory, sandboxed code execution, and production-grade lifecycle management.

🧠 Core Features

Infinite Memory (RAG)

ARC-AI doesn't just keep a chat log — it separates memory into four distinct layers to avoid contamination and keep retrieval sharp:

1. Conversation History
2. Working Context
3. Semantic Memory
4. Long-Term User Facts Retrieval runs through relevance scoring, recency weighting, and duplicate suppression using Pinecone vector search, so the assistant pulls in prioritized, low-noise context instead of dumping everything it knows into the prompt.

🎥 Live Vision & Multimodal Routing

ARC-AI can see. When you speak while your webcam is active, it captures the current frame and pipes it straight into the same real-time socket payload as your voice input, routing visual context to a vision-capable model (Pixtral) with guardrails to prevent silent attachment loss on fallback.

🌐 Live Web Research

Real-time DOM scraping (Cheerio) plus API-based search, weather, and news retrieval — so answers aren't capped at a training cutoff.

⏰ Proactive Routine Engine

Ask it in plain English and it turns that into a scheduled cron job — recurring reminders, deferred tasks, background routines — no manual cron syntax required.

💬 WhatsApp Automation

Full autonomous message composition and delivery through an integrated WhatsApp workflow, so the agent can actually reach you (or others) outside the app.

🖥️ UI Actuation

The assistant can act on its own interface — switching themes, opening websites, playing media, copying to clipboard — driven directly by conversational context.

🏗️ Under the Hood: Provider-Agnostic, Workspace-Isolated Runtime

The architecture went through several major evolutions to get here:

• Provider-orchestrated runtime — an LLMRouter dynamically selects between Gemini (reasoning, multimodal, planning, long-context) and Mistral (fast, cheap, summarization, fallback), while preserving streaming continuity and tool-call consistency across providers.
• Multi-workspace runtime — every workspace behaves as its own isolated environment. Conversations, memories, executions, and even Pinecone vector namespaces are fully scoped per workspace, with zero cross-workspace leakage.
• Execution isolation — the TaskPlanner/TaskExecutor system keeps retries, recovery, and replanning strictly bound to their originating workspace.

- Realtime Socket.IO pipeline — voice or text input flows through ai:stt:final, gets routed for generation, streams token-by-token, and guarantees clean finalization even on interruption.

🔥 What's new in v1.0.0 (first stable release)

The latest release was all about turning ARC-AI from "cool demo" into something you can actually self-host in production:

Security hardening

• Replaced the deprecated, vulnerable vm2 executor with QuickJS WebAssembly sandboxing (quickjs-emscripten) for any code the LLM executes — closing an RCE vulnerability.
• GOOGLE_TOKEN_ENCRYPTION_KEY is now a mandatory env var — the server fails fast on startup if it's missing, instead of silently falling back to something insecure.
• Replaced wildcard CORS with a dynamic origin resolver, fixing credentialed-request validation issues. Runtime reliability
• An activity-aware idle reaper kills headless Chromium (WhatsApp) sessions after 5 minutes of inactivity, so it stops eating RAM/CPU.
• Graceful shutdown on SIGINT/SIGTERM — HTTP servers, MongoDB connections, and Puppeteer/WhatsApp processes all tear down cleanly and in parallel.
• Fixed race conditions between client destruction and recovery hooks. Onboarding & deployment
• Docker Compose now uses portable relative bind mounts instead of hardcoded host paths.

- Full .env.example templates for both root and server, documenting every LLM routing/model/timeout option.

💻 Tech Stack

⚙️ Try it yourself

git clone https://github.com/Aashutosh31/arc-ai-project.git
cd arc-ai-project/server
npm install
cp .env.example .env   # fill in your keys — GOOGLE_TOKEN_ENCRYPTION_KEY is mandatory
npm run dev

Or spin it up with Docker:

cd server
cp .env.example .env
docker compose up --build

Full setup instructions (frontend, Docker, persistent WhatsApp sessions) are in the README.

What's next

I'm continuing to harden the multi-workspace runtime and expand the autonomous tool ecosystem. If you try it out, run into issues, or have ideas, I'd genuinely love the feedback — issues and PRs are open on the repo.

If this is interesting to you, a ⭐ on the repo goes a long way.

🔗 GitHub: github.com/Aashutosh31
🔗 LinkedIn: https://in.linkedin.com/in/aashutosh-bairagi-559aa530b
🐦 X/Twitter: @Aashutosh_dev31
🌐 Portfolio: aashutoshbairagi.vercel.app