Your AI agent skills are probably out of date. Mine were. Here's how I handled it.

My old agent skill was one long file and it told every agent I used to run MinIO for local S3. Then the minio/minio repository on GitHub was archived, and the skill went on recommending it. The agents had no way to know. They just did what the file said.

Instructions for AI agents go stale the same way documentation does, except an agent acts on them without asking whether they're still true. So I rebuilt mine as dev-pops-skills, a set of agent skills built around one idea: every fact in them carries the date it was last checked, and something notices when that date gets old.

In this post:

• what the skills are
• how they stay fresh, which is most of the post
• what happened when I measured whether agents actually use the skills (20 out of 31)

Seven small skills instead of one long file

dev-pops-skills is my DevOps defaults written down as seven small skills: Docker, CI/CD, Terraform, security scanning, shell scripting, whole-project setup, and one for writing and auditing the skills themselves. They work in Claude Code, Antigravity and any agent that supports the open Agent Skills format. An agent loads only the skill that matches the request, which matters because my old single file was so long the agents rarely followed it.

Each skill lists the defaults it applies without asking, such as pinned image tags and GitHub OIDC roles in place of stored AWS keys. It lists what needs a yes first: deleting volumes, pushing images, running terraform apply. And it ends with checks whose output the agent has to paste back, so "done" means something I can read.

Every fact has a date on it

The repo notices when something goes stale, then an agent updates it with me.

Every external fact a skill relies on lives in a sources.md file next to it, with the link it came from and the date it was checked. That covers image tags, action versions, install paths and whether a project still exists. Facts are looked up online when they're written, never from the model's memory. The MinIO fix, for example, is one row:

| MinIO's GitHub repo `minio/minio` is archived. | https://github.com/minio/minio | 2026-09-26 |

Four things watch those facts:

1. Skills expire. Each skill records when it was last fully re-checked, and the validator that runs on every pull request warns once that's more than 90 days ago.
2. A weekly job checks every pin. Every Monday a GitHub Action confirms that each pinned image tag still exists on Docker Hub, and that each pinned GitHub Action exists, isn't archived and isn't a major version behind. On any drift it opens an issue.
3. Dependabot opens pull requests for minor and patch updates to base images. It ignores major versions on purpose: moving the templates to a new Node major changes a default, and that needs a decision from me, not an automatic PR.
4. Real projects report back. When an agent hits a broken or missing template while working in one of my projects, it fixes the project with the smallest change it can, then files a finding in the repo's contributions/ folder with a reproduction. It never names the client, and a CI check fails the build if a name from my private list slips through.

When the weekly issue opens, or findings pile up, I ask an agent to "audit the skills". The skill-authoring skill takes over. It re-checks each fact online and shows me a table of what changed before it edits anything. It applies only the fixes I approve, then bumps the skill's version, adds a changelog entry, updates the dates and opens a pull request. Nothing changes without my approval, and main only accepts a PR that passes five CI checks.

The same loop caught my own tooling: when I moved from Gemini CLI to Antigravity, the install paths changed, and that went in with a source and a date like everything else.

The checks need checking too. I once added two linters set to warn without blocking, so I could fix the existing files first, and they showed a green tick on every run. The logs said both had been failing all along, because GitHub shows a non-blocking failure as a pass. So once in a while I open the logs of the checks themselves instead of trusting the tick.

I checked whether the agents actually use them

A skill only helps if the agent decides to load it, and it decides from a one-paragraph description. So I wrote 31 requests the way I'd phrase them, each paired with the skill that should load, plus four that should load none. A script sends each one to a headless Claude Code that has only these skills and records which skill it loads first.

I didn't want to install the CLI on my laptop for this, so the whole test runs in a throwaway Docker container. The image installs Claude Code from Anthropic's signed apt repository, and the build fails if the signing key's fingerprint doesn't match the one in their docs. I logged in once inside the container, the login lived in a Docker volume, and the repo was mounted read-only so nothing in there could change it:

docker run --rm -v dev-pops-eval-home:/home/eval -v "$PWD:/repo:ro" dev-pops-eval ./scripts/eval-triggers.sh --run

When I was done, I deleted the volume and the image, and my machine was back where it started.

The first full run, on Haiku, got 20 of 31. No request loaded the wrong skill, and all four unrelated ones ("write a React component for a sortable table") correctly loaded nothing. Every miss loaded no skill at all.

Most of those misses were my test's fault. I had asked the agent to "tidy this Makefile" in a project with no Makefile, and it said so, which is the right answer. Each request now runs against a small stand-in project that has the files it mentions. One miss was real: "write a shell script that waits for Postgres" never loaded the scripting skill, and that description is next on my list.

The test also drained my Claude session. I wanted one fact from each run, which skill loaded first, but every request started a full agent session that kept going after it had loaded the skill. It explored the project, read the skill's reference files, and several runs wrote me careful plans for work I would never read. After one full run, I queued two more to measure variance, and eleven prompts into the first of them my five-hour usage window ran dry. A test about whether agents follow instructions had been spending my limit on agents following instructions very thoroughly. Now each run stops the moment a skill loads, gets read-only tools, and is capped at three turns and three cents. By default it runs a sample of nine prompts.

If you want to try it

These are my defaults: Node and TypeScript, Docker Compose, GitHub Actions, and AWS with Terraform. If yours differ, fork the repo and change them; the skill-authoring skill walks an agent through it, and the freshness checks come with it once you enable Actions on your fork (GitHub turns scheduled workflows off on forks).

The repo is poppyszn/dev-pops-skills, MIT licensed. If you found this useful, star it so the next updates show up in your feed, and share the post with whoever on your team keeps the agent instructions. I'd like to hear how other people keep theirs from going stale.