OpenBot's AG-UI Runtime: Per-Agent Browser Isolation and Audit-First Orchestration

OpenBot is a template repository that implements AG-UI (Agent Governance UI) as a runtime layer for multi-agent systems. Each agent gets its own isolated browser session, file system, and tool set. Every action passes through an approval gate before execution and gets recorded afterward. The project positions itself as "the AI assistant your company can actually own" because it runs on your infrastructure and ships as source code, not SaaS.

The repository hit 8,015 stars and trending #5 in TypeScript. It supports any AG-UI-compatible agent, including LangGraph and MCP-based systems. The architecture shows how to build governance into the UI layer rather than bolting it on after the fact.

What AG-UI Means in Practice

AG-UI is a specification for agent interfaces that prioritize human oversight. OpenBot implements it as a runtime that treats agents as coworkers with dedicated channels. Each coworker arrives with:

• A browser session that persists logins and cookies
• A file system scoped to that agent
• A tool set you grant explicitly
• A screen you can watch in real time

The "decided before it happens and recorded after" claim means every tool call hits an approval gate. The human can approve, reject, or take over the session. After execution, the action and result get logged to an audit trail.

This is different from post-hoc logging or LLM-level guardrails. The approval gate sits between the agent's decision and the tool invocation. The agent proposes, the runtime enforces, the human decides.

Architecture: Isolation Without Containers

OpenBot runs on a single machine (your laptop or a dedicated server) and isolates agents at the process and session level, not with VMs or containers.

Browser isolation uses Playwright's browser contexts. Each agent gets its own context with separate cookies, local storage, and cache. Contexts share a browser process but maintain separate state. This is lighter than spinning up full browser instances per agent.

File system isolation uses workspace directories. Each agent writes to a scoped folder. The runtime enforces path boundaries so agents cannot read or write outside their workspace. No chroot or namespace isolation, just path validation at the tool layer.

Tool boundaries are enforced by the AG-UI runtime. When you configure an agent, you grant specific tools. The runtime checks the tool name against the agent's permission set before invoking. If an agent tries to call a tool it does not have, the runtime rejects the call before it reaches the tool implementation.

Approval Gate Data Model

The approval gate is a state machine per tool call. When an agent decides to invoke a tool, the runtime creates an approval request with:

• Agent ID
• Tool name
• Tool arguments (serialized)
• Timestamp
• Request ID

The request enters a pending state. The UI shows the request to the human. The human can:

• Approve: Runtime invokes the tool and records the result
• Reject: Runtime returns an error to the agent
• Take over: Runtime pauses the agent and hands control to the human

After the human decides, the runtime appends the decision and result to the agent's audit log. The log is a JSON Lines file with one entry per action. Each entry includes the request, decision, result, and execution time.

This model works for synchronous tool calls. Asynchronous or background tasks require a different pattern (polling or webhooks) that OpenBot does not currently implement.

Deployment Shape

OpenBot is a monorepo with workspaces for the runtime, UI, and example agents. You clone the repository, configure your agents in the examples/ directory, and run the whole stack locally.

Local setup:

# Clone and install
git clone https://github.com/CopilotKit/OpenBot.git
cd OpenBot
npm install

# Configure environment
cp .env.example .env
# Edit .env with your LLM API keys and agent config

# Start runtime and UI
npm run dev

The runtime starts a Node.js server that manages agent processes and browser contexts. The UI is a Next.js app that connects to the runtime via WebSocket. Each agent gets a channel in the UI where you see its screen (a live view of the browser context) and approve tool calls.

Production deployment is not documented in detail. The repository is in beta and assumes you will adapt it. Likely paths:

• Run on a dedicated server with systemd or Docker Compose
• Use a reverse proxy (nginx, Caddy) to expose the UI
• Store audit logs in a database instead of files
• Add authentication and multi-tenancy if you need multiple users

The "no hosted version" claim means CopilotKit does not operate OpenBot as a service. You run it yourself.

Integration with Agent Frameworks

OpenBot supports "any AG-UI agent." In practice, this means agents that expose their tool calls and accept approval responses. The repository mentions LangGraph and MCP in topics, but the integration layer is not fully documented.

LangGraph integration likely works by wrapping LangGraph's tool invocation with an approval callback. Before LangGraph calls a tool, it hits the AG-UI runtime's approval endpoint. The runtime blocks until the human decides. After approval, LangGraph proceeds with the tool call.

MCP (Model Context Protocol) integration would follow a similar pattern. MCP defines how agents request context and tools. The AG-UI runtime intercepts MCP tool requests, enforces permissions, and logs results.

The key is that the agent framework must support interruptible tool calls. If the framework invokes tools directly without a hook for approval, you cannot plug in AG-UI without modifying the framework.

Observability and Failure Modes

Observability is file-based. Each agent writes to:

• An audit log (JSON Lines)
• A screen recording (optional, via Playwright)
• A console log (stdout/stderr from the agent process)

The UI shows live browser views and recent tool calls. There is no built-in metrics system (Prometheus, Datadog) or distributed tracing. You would add that yourself if you need it.

Failure modes:

• Browser crash: If the shared browser process crashes, all agents lose their contexts. Playwright will restart the browser, but agents lose session state (cookies, local storage).
• Agent process crash: The runtime detects the crash and can restart the agent, but in-flight tool calls are lost.
• Approval timeout: If the human does not respond to an approval request, the agent blocks indefinitely. You need a timeout policy (auto-reject after N seconds).
• Audit log corruption: If the log file is corrupted or deleted, you lose the audit trail. No built-in backup or replication.
• Tool permission bypass: If a tool does not validate paths or permissions correctly, an agent could escape its workspace. The runtime enforces tool allowlists, but the tool implementation must enforce boundaries.

Code Example: Approval Gate Hook

Here is a simplified version of how an agent might integrate with the AG-UI approval gate:

// Agent calls this before invoking a tool
async function requestToolApproval(
  agentId: string,
  toolName: string,
  args: Record<string, unknown>
): Promise<{ approved: boolean; reason?: string }> {
  const requestId = crypto.randomUUID();

  // Send approval request to runtime
  const response = await fetch('/api/approve', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ agentId, toolName, args, requestId })
  });

  // Block until human decides
  const decision = await response.json();

  // Log decision to audit trail
  await appendAuditLog(agentId, {
    requestId,
    toolName,
    args,
    decision: decision.approved ? 'approved' : 'rejected',
    reason: decision.reason,
    timestamp: new Date().toISOString()
  });

  return decision;
}

// Agent uses the hook before tool invocation
async function invokeTool(toolName: string, args: Record<string, unknown>) {
  const approval = await requestToolApproval(this.agentId, toolName, args);

  if (!approval.approved) {
    throw new Error(`Tool call rejected: ${approval.reason}`);
  }

  // Proceed with tool invocation
  const result = await this.tools[toolName](args);

  // Log result
  await appendAuditLog(this.agentId, {
    toolName,
    result,
    timestamp: new Date().toISOString()
  });

  return result;
}

This pattern works for synchronous tools. Asynchronous tools need a different approach (polling or webhooks) to avoid blocking the agent indefinitely.

Technical Verdict

Use OpenBot when:

• You need multi-agent orchestration with human oversight baked into the runtime
• You want per-agent browser isolation without spinning up full VMs
• You can run the stack on your own infrastructure and adapt the template to your needs
• You need audit trails for compliance or debugging
• You are building an internal tool where you control the agent frameworks

Avoid OpenBot when:

• You need a hosted, multi-tenant SaaS solution
• Your agents require container-level isolation for security
• You cannot tolerate single-process failure modes (browser crash affects all agents)
• You need production-grade observability (metrics, tracing, alerting) out of the box
• Your agent frameworks do not support interruptible tool calls

OpenBot is a template, not a product. You will need to adapt it. The AG-UI pattern it demonstrates (approval gates at the runtime layer, per-agent isolation, audit-first design) is the real contribution. If you need that pattern, OpenBot gives you a working starting point. If you need a finished product, look elsewhere.

Source Links

• OpenBot GitHub Repository
• CopilotKit OpenBot Landing Page