Dev.to AI 🤖 Ai 👁 0 📖 7 min read

miyagi: a coding tutor that pauses the YouTube video until your code actually works

This is a submission for the MLH x DEV Writing Challenge What I Built Every coding tutorial breaks at the same moment. The instructor's code runs, yours doesn't, and the video keeps playing. You can paste the

miyagi: a coding tutor that pauses the YouTube video until your code actually works

This is a submission for the MLH x DEV Writing Challenge

What I Built

Every coding tutorial breaks at the same moment. The instructor's code runs, yours doesn't, and the video keeps playing. You can paste the error into a chatbot, but it hasn't seen the video, so it has to guess what you were trying to do.

My teammate Michael Tarekegn and I built miyagi at SanD Hacks to fix that moment. It's a desktop app (Mac, with a Windows build) that turns a YouTube coding tutorial into a hands-on lab. It pauses the video at every step and won't move on until your own code works. The name comes from The Karate Kid: wax on, wax off, and you don't skip ahead until you've actually done it.

Here's how a lesson goes:

  1. Press ⌥Space and type what you want to learn, like "build a neural network." miyagi finds and ranks YouTube tutorials for that goal, or Claude writes a lesson from scratch.
  2. It splits the tutorial into steps, creates a project folder and opens it in your own VS Code.
  3. At each step the video pauses, and a box on your screen marks the exact line to type.
  4. When your screen shows the step working, the video picks back up. If it doesn't, a "Not yet" card compares what you have with what you should have. Hints start as a question and only end with the exact fix, and applying a fix means reviewing the diff and approving it yourself.

Step 1: a box marks the exact line to type on, and the drawer has the code.

We split the work down the middle. Michael built the engine: video search, lesson building, screen reading, step grading and a small VS Code companion extension. I built the desktop app you actually see: the overlay that lives under the MacBook's notch, the launcher, the lesson drawer, the spotlight box and the cards, plus the Windows build, the brand and miyagi.study. The two halves talk over a WebSocket through a shared event contract, so we could build in parallel all weekend without stepping on each other.

What I learned:

  • Speed and accuracy only fight when every check costs the same. Most of our checks are cheap and certain: does the file have this function, do the shapes match, did the loss go down. A model only gets called for real judgment calls, and it's never allowed to overrule a check that failed.
  • Ask the source before guessing from pixels. We started with OCR on screenshots, and it flickered between verdicts and misread UI. Reading the editor's text directly was faster and exact.
  • Test on the real machine early. Almost every overlay bug only showed up on an actual Mac.

Demo

The demo lesson is a small NumPy neural network that learns to draw sin(x). We planted a bug in step 2: the slope of tanh has the wrong sign. It passes the shape check, so nothing looks wrong until you train and the loss gets stuck.

The loss is stuck at 0.49, so the step stays open and the card asks why.

miyagi keeps the step open, names the bug ("Wrong sign in tanh slope") and starts with a question instead of the answer. After the fix, the loss falls from 0.82 to 0.0009, and a real run in VS Code prints -1, 0 and 1. On a real Mac, the coach named the planted bug about 7 seconds after training finished.

Training passed: the loss fell from 0.82 to 0.0009 while the network trained on screen.

It isn't a one-tutorial trick. We pointed it at five tutorials outside the demo (React, git, pandas, Rust and FastAPI). Each became a valid lesson, built live in 24 to 65 seconds, and all 43 step titles matched their transcripts. In a physics-informed neural network lesson, the coach worked out without being told that a learning rate of 0.3 was why the loss was stuck.

Partner Technologies

Bright Data: finding tutorials that still work

Finding the right tutorial is half the problem, and plenty of popular ones have quietly broken since they were recorded. Claude searches the web for candidates, Bright Data's SERP API pulls Google's video results for the learner's goal, and Claude scores each video against that goal. Then Bright Data's Web Scraper API reads each video's top comments. The comment section is where people report "this doesn't work anymore," so miyagi can flag a dead tutorial before you sink 40 minutes into it.

Watch a video: YouTube tutorials found and ranked for your goal.

Gemini: watching the video

Gemini watches the tutorial itself and splits it into steps, with the code each step should produce. That matters because in a coding video, the code is on screen, not in the transcript. In parallel, Claude builds a lesson from the transcript. If Gemini hasn't answered within 75 seconds, the Claude version takes over, so the learner is never stuck staring at a spinner.

Tiger Data: every step and loss value as a time series

Every step event, check result and training-loss value is a row in a Tiger Data hypertable (learning_events and training_metrics). Continuous aggregates keep the live loss curve and per-step stats precomputed, and they drive a stats page in the app. The run you just finished shows up there right away, and across learners the same tables show where people get stuck in a video. We picked Tiger because this is time-series data, and because it's still Postgres, runs and lessons join with it like any other table.

The venue Wi-Fi blocked Tiger Cloud's port. The engine already wrote to a local spool when it couldn't connect and retried every 30 seconds, so nothing was lost. On Sunday morning more than 1,100 rows were waiting; with a VPN on, they uploaded in one go and the stats page came back online before judging.

Claude: lessons, coaching and the build itself

Claude writes lessons when there's no good video, scores tutorials and does the coaching: the "Not yet" cards and the hint ladder that goes from a question to the exact fix. We also wrote most of miyagi with Claude Code, running several sessions in parallel, each in its own git worktree. With that many agents merging, we added a Grok-powered "Repo Steward" bot that checked new merges every 20 minutes for contract changes, edits that crossed into the other person's folder, and demo risks.

Our Grok bot reviews every merge for contract changes, lane crossings and demo risks.

Hackathon Experience

SanD Hacks is UC San Diego's student hackathon (November 14–15). Michael and I came as a team of two, and we split ownership on day one: he owned the engine, I owned the app, and the steward bot flagged any change to the event contract between them. With so many Claude Code sessions running, our agents ended up coordinating by email. My inbox from Sunday morning is full of status reports signed "Michael's Claude, for Michael" and "Adrian's Claude, for Adrian."

Here's what broke along the way:

  • Screen capture on macOS 26. The ScreenCaptureKit connection kept dropping, and the CoreGraphics fallback hung. We switched to Apple's screencapture tool.
  • Reading the screen. OCR flickered between verdicts, and asking a model where to draw the box was accurate but slow and expensive. Now the VS Code extension reads the editor's text directly and macOS Accessibility data gives exact positions. OCR is only a fallback.
  • Noisy loss curves. Training loss is noisy enough that a stuck run often looked like it was "improving" by chance. Rank-based trend tests (Mann-Kendall and Theil-Sen) cut false "improving" verdicts from as many as 136 of 300 curves to 3 or fewer.
  • The overlay. miyagi is a full-screen window on top of everything. It has to take clicks on its own controls, pass every other click through to your editor, and never steal your keyboard focus. When it does end up in front with nothing focused, it hands focus back to whatever app you were last in.
  • Sunday, 9:04 AM. A commit added import { execFile } from 'node:child_process' to the top of a main-process file that the renderer also imported through a bridge module. Vite externalized child_process for browser compatibility, and every window threw on load: overlay, launcher, player and settings. The app came up with nothing on screen. The fix was a dynamic import inside the one main-process function that needed it, and it was merged by 9:25.

A judge told us they weren't watching the Devpost videos, so the live demo at our table was what counted. I did the intro and the close, Michael drove the lesson live, and we won the Grand Prize.

What I'll remember most is that 9:04 crash: a few hours before judging, every window dead, and the fix was moving one import. That's the kind of moment miyagi is built for, except this time we were the ones stuck at a broken step.

Next, we want screen reading on Windows, a dashboard that shows tutorial creators where learners get stuck, and lessons beyond the code editor for any tool that prints to a terminal. Longer term, we want to use a camera to guide hands-on tasks that aren't coding at all.

If you try miyagi, I'd love to hear where it gets stuck.

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.