Dev.to AI 🤖 Ai 👁 0 📖 9 min read

Kill -9 an Agent Mid-Commit and .git/index.lock Outlives It. How Git's Locks Work

Book: Database Playbook Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code a

Kill -9 an Agent Mid-Commit and .git/index.lock Outlives It. How Git's Locks Work

Your coding agent was halfway through git commit -am "fix login" when its turn hit a hard timeout and the process got killed. You start a fresh session, ask for the same commit, and git refuses before it does anything:

fatal: Unable to create '/srv/app/.git/index.lock': File exists.

That first line is built from a format string in git's lockfile.c. Recent git adds one more sentence: "Another git process seems to be running in this repository, or the lock file may be stale." There is no other git process. The one that created the file is gone, and it died in the one way that gave it no chance to clean up.

The fix everyone knows is rm .git/index.lock. It is usually right. To know when it is right, and why parallel agents in separate worktrees can still trip over each other's locks, you need the protocol behind that file.

How git takes a lock

Git never edits the index, a ref or the config file in place. The comments in lockfile.h spell out the steps:

  1. Create <file>.lock with O_CREAT|O_EXCL, "so that we can notice and fail if somebody else has already locked the file." Per open(2), that combination makes open() fail with EEXIST if the path already exists, so only one process can win.
  2. Write the new contents into the .lock file, never into the real one.
  3. Commit by renaming the .lock file over the original. rename(2) replaces the target atomically, "so that there is no point at which another process attempting to access newpath will find it missing."
  4. Or roll back by deleting the .lock file.

Readers never take the lock. They see the old file or the new one, never half of each.

The cleanup half lives in tempfile.c. The first time git activates a temp file, it registers an atexit handler and a signal handler through sigchain_push_common. sigchain.c shows which signals that covers: SIGINT, SIGHUP, SIGTERM, SIGQUIT and SIGPIPE. The handler deletes every active temp file, restores the previous handler and re-raises the signal so the process still dies the way it was told to.

SIGKILL is not on that list, and it can't be. The signal(7) man page: "The signals SIGKILL and SIGSTOP cannot be caught, blocked, or ignored." The kernel removes the process, no handler runs, and index.lock stays on disk.

So the signal decides the outcome. Ctrl+C sends SIGINT, plain kill sends SIGTERM, and in both cases git cleans up. kill -9 leaves the lock behind. So does anything that escalates to SIGKILL: docker stop sends SIGTERM to the container's main process and SIGKILL after a grace period of 10 seconds on Linux containers. A pre-commit hook that runs your test suite can easily outlast that.

Editorial illustration of a black door with a burnt-orange padlock left hanging on it after the figure who locked it has vanished, only footprints leading away
A lock taken with O_EXCL only goes away if the process that took it is alive to remove it.

The stale file is also harmless to your data. The rename in step 3 never happened, so the real .git/index is exactly what it was before the dead command started. Whatever the killed commit had staged lives only in that stale lock file, and removing the lock discards it. Your working tree is untouched, so nothing is lost: you stage and commit again.

Reproduce it in under a minute

One detail matters here: plain git commit does not hold the lock while your hook runs. In builtin/commit.c, the as-is path writes the index with COMMIT_LOCK | SKIP_IF_UNCHANGED, which releases the lock before the hook starts. The -a path writes the index "still locked" and returns the lock file's path, with a comment that says "The caller should run hooks on the locked real index." That is the path git commit -am takes, so that is the one to test.

Run this in a scratch directory. It assumes your git identity is already set.

mkdir lockdemo && cd lockdemo
git init -q
echo one > a.txt
git add a.txt
git commit -qm init

cat > .git/hooks/pre-commit <<'EOF'
#!/bin/sh
sleep 60
EOF
chmod +x .git/hooks/pre-commit

echo two >> a.txt
git commit -am lockdemo &
sleep 2
ls -l .git/index.lock
pkill -9 -f 'commit -am lockdemo'

The first ls finds the lock, because git is parked inside the hook. Now look at what the kill left behind:

ls -l .git/index.lock
git add a.txt
pgrep -fl pre-commit

The lock is still there, and git add fails with the Unable to create error from the opening. The pgrep line is the part people miss: kill -9 hit git and nothing else, so the hook process is still sleeping. In a real repo that orphan might be your linter or test runner, still reading and writing files after the agent that started it is gone.

Clean up in the right order: stop the orphan, then remove the lock.

pkill -f 'hooks/pre-commit'
rm .git/index.lock
git status --short

a.txt should show as modified but unstaged, which is what step 3 promised: the real index never saw the dead commit's changes.

Make git tell you the lock is stale

Git 2.54 added a way to record "process ID of the process that holds the lock next to a lockfile for diagnosis." It is off by default. The core.lockfilePid docs explain the format: the PID file sits next to the lock with ~pid inserted before .lock, so index.lock gets an index~pid.lock holding one line, pid <value>.

git config --global core.lockfilePid true

When a later lock attempt fails and that PID file exists, unable_to_lock_message in lockfile.c probes the PID with kill(pid, 0). If the process is gone you get "Lock was held by process N, which is no longer running; the lock file appears to be stale." If something answers, git says the lock "may be held" and warns that "PIDs can be reused." Git still does not delete the file for you. On a machine where agents get killed on timeouts, that one config line turns a guess into a diagnosis.

The safe removal routine, with or without the PID file:

  1. pgrep -fl git and make sure nothing is running in this repo, including hooks.
  2. git rev-parse --git-path index.lock to get the real path. In a linked worktree it is not .git/index.lock.
  3. Remove that one file and nothing else.

Which locks a worktree owns, and which it shares

Parallel agents often get one git worktree each, and the docs are precise about what that isolates. Each linked worktree has a private directory under .git/worktrees/<id>, which becomes its $GIT_DIR. $GIT_COMMON_DIR points back at the main .git. git rev-parse --git-path resolves each file against one or the other.

"In general, all pseudo refs are per-worktree and all refs starting with refs/ are shared," with exceptions for refs/bisect, refs/worktree and refs/rewritten. HEAD is per worktree. Per gitrepository-layout, packed-refs and config come from $GIT_COMMON_DIR when it is set, so they are shared too. Only extensions.worktreeConfig gives each worktree its own config.worktree.

File Lock file Who shares it
index .git/worktrees/<id>/index.lock one worktree
refs/heads/<branch> refs/heads/<branch>.lock all worktrees
refs/remotes/origin/* <ref>.lock all worktrees
packed-refs packed-refs.lock all worktrees
config config.lock all worktrees

Two agents committing on their own branches in their own worktrees don't collide: different index locks, different branch ref locks. They collide when they touch the same shared file at the same moment:

  • Both run git fetch. Each wants refs/remotes/origin/main.lock. refs/files-backend.c wraps the lock failure as "cannot lock ref '%s': %s", so the loser reports the ref name followed by the same Unable to create text.
  • One runs git push -u or git branch --set-upstream-to while another does the same. Both write the shared config. A 2019 BitBake patch for a fetcher race noted that git "only tries exactly once" on .git/config.lock.
  • One deletes a packed branch while another runs git gc or git pack-refs. Both need packed-refs.lock.

A stale lock on any shared file blocks every worktree, not only the one whose agent died.

Editorial illustration of several small black desks, each with its own burnt-orange notebook, all connected by strings to one shared filing cabinet in the center with a single padlock
Every worktree gets its own index. Refs, packed-refs and config are one shared cabinet with one lock per drawer.

Live contention has a knob. Ref locks retry for core.filesRefLockTimeout, 100 ms by default. The packed-refs lock retries for core.packedRefsTimeout, 1000 ms by default. The retry loop in lockfile.c uses quadratic backoff with jitter. On a machine running several agents, give them room:

git config --global core.filesRefLockTimeout 2000
git config --global core.packedRefsTimeout 5000

That helps when another process really holds the lock and is about to release it. Against a stale lock it only makes git wait longer before failing.

reftable redraws the map

Everything above is the default "files" backend: one file per ref, one lock per ref. The reftable backend stores refs in a stack of tables listed in reftable/tables.list. Every update acquires tables.list.lock, and the spec is blunt: "the repository is single-threaded for writers." Writers busy-spin with backoff around that lock and abort if the repository stays too busy.

In a reftable repository, two agents committing to different branches in different worktrees queue on the same lock file. By the spec's own logic, a lock left behind on tables.list stops ref updates in every worktree at once. In exchange you get the advantages Git's BreakingChanges page lists, including atomic multi-reference writes and no case-collision problems on case-insensitive filesystems. That page says Git 3.0 will make reftable the default for new repositories, and that "there is no planned release date" yet. You can opt in today through init.defaultRefFormat.

Where Hermes fits

I build Hermes IDE, so weigh this section with that in mind. In Hermes, ⌘N (Ctrl+Shift+N on Windows and Linux) starts each task on its own hermes/<name> branch in its own worktree. In the table's terms, every agent gets its own index and its own index.lock, so one agent killed mid-commit does not stop another agent's git add.

The other half is how agents end. On macOS and Linux, agents keep working while Hermes quits, updates or crashes, and you reattach where they left off. There, closing the app is not a kill -9 for an agent sitting inside a pre-commit hook.

The Review Desk's Worktrees tab in Hermes IDE, listing one worktree each for orders-api, web-app and orders-types with branch, task, age and disk size
The Review Desk's Worktrees tab: one Hermes worktree each for orders-api, web-app and orders-types under ~/code, with each worktree's hermes/ branch, task, age and disk size, and the free disk space.

What it does not change is git. Refs, packed-refs and config are still shared across worktrees, so two agents fetching at once can still hit cannot lock ref. Nothing in the Hermes release notes says it removes stale lock files, so the routine above still applies. Hermes is free, source-available under BSL 1.1, runs on macOS, Windows and Linux, and is not affiliated with Nous Research, Anthropic, OpenAI or any agent vendor. Download it or read the code on GitHub.

Have you found a stale lock that wasn't index.lock? Tell me which file it was in the comments.

If this was useful

Git's ref store is a small database with two storage engines: one file per key with a lock per key, or a stack of immutable tables behind a single writer lock. Picking between those is the same kind of trade you make when you pick a store for your own system: who can write at once, what a crash leaves behind, what a reader sees mid-update. The Database Playbook is my short guide to making that call for your own data.

Database Playbook: Choosing the Right Store for Every System You Build

📰 Read the original article on Dev.to AI

Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.