Five Worktrees, Five node_modules: How Parallel Agents Exhaust Linux's inotify Limit

• Book: System Design Pocket Guide: Fundamentals
• The series: Pocket Guides for Developers — 8 short books on agents, RAG, prompts, observability, events, databases and system design — the whole series
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

ENOSPC: System limit for number of file watchers reached. The fourth of your five coding agents prints that when its dev server boots on a Linux box. It is the same message a Cursor user posted in June when Vite refused to start.

Nothing about the setup is unusual. Each agent has its own git worktree, each worktree ran npm ci, and each agent starts a dev server to check its work. You run df -h and the disk is half empty. ENOSPC is the "no space left on device" errno, but here it has nothing to do with disk. The man page for inotify_add_watch(2) reuses it for a different wall: "The user limit on the total number of inotify watches was reached or the kernel failed to allocate a needed resource."

Hermes IDE 2.2.9 shipped a fix for its own share of this problem. I build Hermes, so that part comes last and you can weigh it accordingly.

One watch per directory, one budget per user

inotify(7) is how Linux tells a program that files changed. A program adds a watch on a path and reads events from a file descriptor. The man page is blunt about the part that matters here: "Inotify monitoring of directories is not recursive: to monitor subdirectories under a directory, additional watches must be created."

So a tool that wants to know about every change under a project root walks the tree and adds one watch per directory. Files don't each need one, because a watch on a directory reports changes to the files inside it. The number that grows with your project is the directory count.

Node's fs.watch works this way too. Its docs say that on Linux "this uses inotify(7)", and its recursive option, added for Linux in v19.1.0, has to do that same walk underneath.

The second half of the rule is in the same man page. max_user_watches "specifies an upper limit on the number of watches that can be created per real user ID." Per user, not per process. Your five agents, their five dev servers, your editor, its language servers and any test runner in watch mode all run as you, so they all draw from one pool.

Now add worktrees. Git shares the history between worktrees, "everything except per-worktree files such as HEAD, index, etc." Ignored folders like node_modules aren't shared at all, so every worktree that installs its own dependencies gets its own copy of the tree. Hard links don't rescue you: link(2) fails with EPERM when "oldpath is a directory", so each copy is a fresh set of directories. Five worktrees means roughly five times the directories for anything that watches them.

Every worktree brings its own dependency tree, and every directory in it is a watch drawn from one per-user pool.

What your limit is set to

For most of inotify's life the default was 8192. A kernel change by Waiman Long replaced it with a value sized from memory. It was merged in December 2020 with the fsnotify pull for v5.11-rc1 and shipped in Linux 5.11. The commit message notes that 8192 had been the default since inotify arrived in 2005. Here is the computation today, from fs/notify/inotify/inotify_user.c:

watches_max = (((si.totalram - si.totalhigh) / 100)
        << PAGE_SHIFT) / INOTIFY_WATCH_COST;
watches_max = clamp(watches_max, 8192UL, 1048576UL);

That is 1% of low memory divided by an estimated cost per watch (the size of an inotify mark plus two inodes), clamped between 8192 and 1,048,576. The commit message says a 64-bit system with 128 GB or more will likely get the maximum. Below that the value scales linearly with RAM, so a 16 GB machine lands somewhere around an eighth of the cap, roughly 131,000.

On top of the kernel default, distros and packages often set their own. The Cursor report above was on a machine at 65,536. VS Code's Linux docs suggest 524,288 and note that "each file watch takes up 1,080 bytes", about 540 MiB if every one of those watches is used. So skip the guessing and read the number your kernel is actually using:

cat /proc/sys/fs/inotify/max_user_watches

Count it yourself

Two numbers decide whether five worktrees fit: how many directories a naive recursive watcher would cover, and how many watches your processes hold right now. Both are a few lines of bash.

The first one walks every worktree of the current repository. git worktree list --porcelain prints one worktree <path> line per worktree:

#!/usr/bin/env bash
limit=$(cat /proc/sys/fs/inotify/max_user_watches)
total=0
while read -r key wt; do
  [ "$key" = worktree ] || continue
  n=$(find "$wt" -type d | wc -l)
  printf '%8d  %s\n' "$n" "$wt"
  total=$((total + n))
done < <(git worktree list --porcelain)
echo "dirs: $total  limit: $limit"

Swap "$wt" for "$wt/node_modules" to see how much of each count is dependencies. The main worktree's count also includes its .git directory, which a watcher with no exclusions would cover too. Treat the total as a ceiling: well-behaved tools skip node_modules and build output, so they use far fewer.

The second number is what's in use. Each inotify watch shows up as one inotify wd: line in the process's fdinfo, as documented in proc_pid_fdinfo(5). This is the same method the Cursor reporter used to find their two biggest consumers:

#!/usr/bin/env bash
for p in /proc/[0-9]*; do
  n=$(cat "$p"/fdinfo/* 2>/dev/null | grep -c '^inotify wd:')
  [ "$n" -gt 0 ] && echo "$n $(cat "$p/comm")"
done | sort -rn | head

You can only read fdinfo for your own processes without root. That's fine here, because the limit is per user and your processes are the ones spending it. I haven't run these on any particular project, so there are no numbers here. Run them on yours before and after adding a worktree.

Why your Mac never hits this wall

Run the same five agents on macOS and the dev servers come up. Node's docs explain the difference in one line: on macOS, fs.watch "uses kqueue(2) for files and FSEvents for directories."

Apple's File System Events overview describes the model. You register once for a directory hierarchy, and "the file system events daemon will post a notification every time that any file in the monitored directory hierarchy changes." One registration covers the whole tree, so the number of directories under it doesn't turn into a count of kernel watches charged to your user.

The trade-off is precision. The same page says "the granularity of notifications is at a directory level. It tells you only that something in the directory has changed, but does not tell you what changed", and that the daemon coalesces bursts of changes. Tools on macOS re-scan the directory to find out what happened. On Linux they get exact per-file events and pay for them one directory at a time.

FSEvents watches a hierarchy with one registration. inotify needs a watch on every directory in it.

The fixes, cheapest first

Stop watching what nobody edits. Most tools already exclude dependency folders by default. Vite "skips the .git/, node_modules/, test-results/, and Vite's cacheDir and build.outDir directories by default." VS Code's default files.watcherExclude "excludes node_modules and some folders under .git." webpack's watch docs show the one-liner for its watcher:

export default {
  watchOptions: {
    ignored: /node_modules/,
  },
};

In the Cursor report, adding files.watcherExclude entries for node_modules, target, dist, build, .git/objects, .cargo and a few project-specific folders dropped the total from about 65,532 watches to about 193 on the same workspace. Check every watcher in your setup, including the dev server each agent starts. One tool without exclusions can eat the pool for all the others.

Raise the ceiling when you need to. If your counts say the work genuinely needs more watches, set the value in a sysctl drop-in. VS Code and webpack both suggest 524,288:

echo 'fs.inotify.max_user_watches=524288' \
  | sudo tee /etc/sysctl.d/90-inotify.conf
sudo sysctl --system

Remember VS Code's figure of 1,080 bytes per watch. A higher limit is a promise of kernel memory if something does use it, so on a small VM pick a lower number, and do the exclusions first.

What changed in Hermes 2.2.9

Hermes IDE runs Claude Code, Codex and other agents side by side, and each task can get its own branch and worktree. That puts a lot of worktrees on one machine, each with whatever dependency and build folders the project has. Before 2.2.9, Hermes itself was one of the consumers drawing on that pool.

The 2.2.9 release, published October 12, 2026, puts it this way: "Hermes no longer slows down or uses up the system's file-watch limit when worktrees contain large dependency or build folders." The notes don't say how the fix works. On a per-user limit, the outcome is what matters: watches Hermes no longer holds stay available to your agents' dev servers and test runners.

The same release also retries removing a worktree that couldn't be deleted when its session closed, at the next launch. Fewer leftover worktrees means fewer copies of node_modules lying around for anything to walk.

I build Hermes IDE, so weigh this section with that in mind. It's free, source-available under BSL 1.1, and runs on macOS, Windows and Linux. Whichever tool you run parallel agents in, the fdinfo script will name the process spending your pool.

If this was useful

A per-user watch limit is a shared budget that many processes draw on and nobody owns. Measuring who spends it before you decide between cutting demand and raising the ceiling is the habit I write about at system scale in System Design Pocket Guide: Fundamentals. It's part of the Pocket Guides for Developers series: 8 short books on agents, RAG, prompts, observability, events, databases and system design.

If you want to try parallel agents in worktrees, download Hermes IDE or read the source on GitHub.