Giving an AI agent eyes on a running React Native app
Over the last few months I've posted small findings about AI agents, MCP and React Native internals. They were all side effects of building one tool. This post is about the tool: Parallax, and the problems that shaped it
Over the last few months I've posted small findings about AI agents, MCP and React Native internals. They were all side effects of building one tool. This post is about the tool: Parallax, and the problems that shaped it.
The problem
An AI agent working on a React Native app sees source files and a terminal. It doesn't see:
- the screen. "Fix the padding on that button" means nothing without it.
- which component is which. The rendered tree is mostly library wrappers.
- runtime errors. In 90 seconds of an app running, the simulator's native log had 3,498 events, and 0 of them came from the app, React Native or Hermes. JS errors live in the JS runtime, not in os_log.
So the agent guesses, and you end up copying logs and describing screenshots for it.
What Parallax is
Two halves that share one live connection to the app:
-
An MCP server for the agent (Claude Code and Codex). Tools for the screen tree with
file:line, console logs, errors, screenshots, taps, swipes, typing, reload. - A desktop window for the human. The live simulator screen with real input, Select element β an inspector with the component, props and source line β Send to agent, approval cards for the agent's device actions, two devices side by side.
Setup is one command per agent:
npx -y @parallax-rn/mcp@<version> setup claude
npx -y @parallax-rn/mcp@<version> setup codex
It shows a diff of what it will change and asks before writing anything. setup ... --undo removes it.
Things that turned out harder than expected
Element β source line. React 19 removed _debugSource. The location now lives in _debugStack, which you symbolicate through Metro. The catch: for elements inside .map(...), the line points to the start of the callback, not the JSX tag (8 of 22 points in my test, with the tag 14β24 lines lower). Parallax searches forward for <ComponentName up to 40 lines.
The tree is mostly noise. On a real screen: 209 rows, 126 of them wrappers (View_withRef 57, SVG parts 45, PressabilityDebugView 23). The window folds them unless a row has a testID, label, text or handler, leaving 83. The agent still gets the full tree: hiding things from the model is a different decision than hiding them from a human.
Unhandled rejections hide. In older RN they're a console.warn; in RN 0.87 a console.error via ExceptionsManager. In both cases the global ErrorUtils handler never sees them, so a tool that only hooks there misses all of them.
A tap is fast, checking it is slow. device_tap takes 1.3β2.3 s end to end, but the tap itself is 56 ms. The rest is reading the accessibility tree and waiting for the screen to settle. If you want a faster agent loop, speed up verification, not input.
One simctl list doubled click latency. Adding a second simulator put a full xcrun simctl list (~100 ms) on the pick path. Click β inspector went from 77 ms to 156 ms p50, and nobody noticed for a whole step because that check measured a different number. Now every number has one definition and a threshold in the script.
Why there are approvals
While testing, an agent crashed the simulator's backboardd three times on purpose "to confirm the diagnosis". It wasn't malicious, just thorough. Since then the agent's device actions can go through approval cards in the window, with a strict mode if you want every action confirmed. Secret-looking props (authToken, apiKey) are masked in the inspector, matched by camelCase/snake_case words so passenger and authorName stay readable.
Some numbers
| What | Result |
|---|---|
| Click on screen β inspector ready | 77 ms p50 |
| Component pick at a point | ~4β5 ms |
| Live stream | 44β52 fps (screenshots: 2.3 fps) |
| MCP cold start via npx | 4.3 s β ~1.5 s (one bundled file) |
| Tool descriptions in context | ~2,500 tokens, guarded by a test |
What it is and isn't (yet)
- macOS; iOS simulator and Android emulator. No real devices yet.
- Bare React Native first; Expo works.
- Runs locally. No account, nothing about your app leaves your machine.
- Free beta. The agent tools stay free.
Try it
The beta is out and free: https://parallax-rn.dev/ (install guide: https://parallax-rn.dev/docs/getting-started/install). The app is also a direct download from GitHub: https://github.com/parallax-rn/parallax-releases/releases/tag/v0.1.0-beta.2.
If you work on React Native with Claude Code or Codex, I want to know where it breaks for you, and the site has a short feedback form for that.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.