I turned my Obsidian vault into a 3D hologram I control with my hands

My Obsidian vault has around 2,000 notes. Obsidian's built-in graph view is nice, but I wanted something I could grab: a 3D "second brain" I could rotate with my hand, like the holograms in Iron Man.

So I built it. It is free, it runs entirely in your browser, and you can try it without installing anything:

Live demo: https://yojoxsn.github.io/vault-3d/
Source (MIT): https://github.com/YojoxSn/vault-3d

Captured from the live demo with mouse and keyboard.

I did not write the code by hand. I described what I wanted to Claude Code in a long, structured prompt, then iterated. This post covers what the tool does, how it works, and the bugs that were worth the trouble.

What it does

• Orb mode. Every note is a glowing sphere on a glass globe. Height is time (newest notes at the north pole), each root folder gets its own wedge, and well-connected notes sink toward a glowing crystal at the center. Notes that are linked but do not exist yet show up as wireframe "ghosts".
• Insights. A panel that computes four lists from your graph: your backbone (most connected notes), notes to write next (ghost notes linked most often), orphans to connect, and well-connected notes you have not edited in a long time. This turned out to be the most useful part.
• Hand control (experimental). Open hand rotates, pinch clicks, two pinched hands zoom, a fist freezes the rotation, a swipe goes to the previous or next linked note. The video never leaves the browser tab.
• Share card. One button saves a picture of your own vault with its numbers. It contains only counts, never note titles, so it is safe to post.
• Private by design. Drop your vault folder on the page: it is read inside your browser, read-only, and nothing is uploaded.

The prompt

The whole project started from a prompt written like a spec: context, numbered steps, concrete deliverables, and a few hard constraints. The shape was:

Create an interactive 3D visualization of ALL my Obsidian vault with Three.js,
controllable with my hands through the webcam, like Tony Stark's hologram.

Step 1 - Import the vault: parse [[wikilinks]], aliases, markdown links, tags,
resolve links like Obsidian does; links to missing notes become "ghost" nodes.
The vault stays READ-ONLY.
Step 2 - 3D scene: sphere per note (size = connections, color = folder),
glowing links with particles, d3-force-3d layout. It must stay smooth with 3000+ notes.
Step 3 - Hand control with MediaPipe: open hand = rotate, pinch = click, ...
Everything must also work with mouse and keyboard.
...
At the end, give me the exact commands to install and run it.

Three habits made the biggest difference: name the constraints ("read-only", "3000+ notes", "works without a camera"), ask for tests, and ask the assistant to say what it could not verify. The full original prompts (in French) are in the repo: PROMPT.md.

How it works

Parser. A pure function turns markdown files into a graph. It handles [[Note]], [[Note|alias]], [[Note#heading]], embeds, [text](Note.md), inline and frontmatter tags, and resolves names the way Obsidian does (case-insensitive, closest folder wins). Because it is pure, the exact same code runs in the browser (drop a folder) and in a Node script (VAULT_PATH). On a synthetic vault of 2,000 notes and 16,000 links it takes about 0.2 s.

Rendering. Spheres are one InstancedMesh with a custom shader (fresnel rim, pulse, fog). All links are a single LineSegments. The particles that flow along the links are animated in the vertex shader, so there is no per-frame CPU work. Bloom plus a small post-processing pass (scanlines, chromatic aberration, grain) gives the hologram look.

Layout. d3-force-3d runs in a Web Worker so the UI never freezes; for 2,000 notes it needs about 5 s while the nodes stream in. The orb layout is a pure function instead of a simulation: latitude comes from the rank of the modification date, longitude from the folder wedge, and the radius shrinks with the number of connections.

Hands. MediaPipe's HandLandmarker gives 21 points per hand. They go through a One Euro filter, then a pure gesture recognizer (open, fist, point, pinch with hysteresis), then a small state machine that handles debouncing, "quick pinch = click, held pinch = grab", the two-hand zoom, swipes and the fist-hold. Keeping the recognizer free of any DOM or MediaPipe code meant I could unit-test it with simulated hands, no camera needed.

Bugs that were worth it

• A click became a drag. When you release a pinch, the thumb moves away and the midpoint between thumb and index drifts. The cursor moved, the "click" was rejected as a drag, and it would have landed in the wrong place anyway. The fix: track the index fingertip, which barely moves, and remember where the cursor was when the pinch started. A unit test caught it.
• Left/right hand labels are a trap. Handedness labels depend on whether the video is mirrored. For the "both palms toward the camera" gesture I stopped trusting them and infer left vs right from the hand's position in the image instead.
• A YAML regex swallowed a newline. Tags written as a block list (tags: then - a) were read as the tag -. Found by a test, fixed in one line.
• One unreadable file broke the whole import. An undownloaded iCloud placeholder throws when read. Now a single bad file is skipped and reported.
• "It stutters on my machine." My first version looked great and ran badly on some graphics cards. The cost was mostly the full-resolution bloom and the glass-blur CSS over the animated canvas. There are now three quality levels, and Auto lowers the quality by itself when the frame rate stays under 24 fps. On the software renderer I used for testing, the levels went from about 1.6 fps (high) to 7 (medium) to 18 (low), so the levels genuinely matter.

What I could not verify

I want to be straight about the limits:

• The gesture thresholds are tested with simulated hands and in a real browser pipeline, but webcams, lighting and hands vary a lot. Real-world tuning is not done, and I would love reports.
• I measured parsing and layout on 2,000 notes, but not the frame rate on a real GPU at that size.
• The hand-control clip is the part I have not recorded yet, so the GIF above uses the mouse.

Try it, break it

1. Open https://yojoxsn.github.io/vault-3d/ and press H for hands mode (good light, 40-80 cm from the camera).
2. Drop your own vault folder on the page and look at the Write next list.
3. Press P to save a share card of your vault.

If it helps you, a ⭐ on the repo is the best way to help others find it. If something breaks, open an issue with your browser and webcam. And if you build your own version from the prompt, I would like to see what you get.