Uptime dashboards do not tell you when a model starts drifting

We had good monitoring: pager alerts, latency graphs, error budgets, a dashboard someone actually looked at. What we did not have was any way to notice the model was getting worse.

That is a category error, not a contradiction. Everything on the dashboard was about whether the service was up. Article 72 of the EU AI Act asks for something else: a post-market monitoring system that systematically collects and analyses performance data across the lifetime of a high-risk AI system. The point is to detect what changes after you ship, not what breaks at deploy time. A green dashboard and a quietly drifting model are entirely compatible.

Two kinds of monitoring that share a name

Post-market monitoring is not observability with a different label. Observability asks whether the system is running, fast, and free of errors. Post-market monitoring asks whether it still behaves as intended, whether new risks are emerging, whether it is used in ways we did not predict, and whether documented controls are still adequate.

Those need different signals: user complaints, support tickets, incident reports, override rates, distribution shifts in inputs, disagreement with a human reviewer, outcomes in the cases the system touched. None show up in an error budget, and most are not numbers until someone decides to collect them, which is why they tend not to exist. I also got the loop wrong. Article 72 is not a standalone log of complaints; it feeds back into risk management under Article 9. A signal nobody triages, that never updates the risk file, and never changes a control is just a mailbox, and I had a mailbox.

The three things a plan actually needs

Written by hand, the plan reduced to three decisions: which signals we watch and where each comes from, the cadence at which someone reviews them and who owns that review, and the documented path by which a finding reaches risk management and becomes a change. Signals are a scoping problem: "monitor the model" is not a signal, and you have to pick a small set you can instrument and admit what you are not watching. Cadence is an ownership problem: a review with no named owner and no interval happens once. The feedback loop is the part most plans omit, because drawing an arrow into the risk file forces you to say who updates it and on what trigger.

I built PmmPack to draft those three decisions. You paste how you run a system in the market, and it returns an Article 72 post-market monitoring starter: the signal themes, a review cadence, the feedback into the risk management system, and the evidence gaps in the plan. That is all it does. It does not collect a signal, connect to your dashboards, or run a review.

A plan starter is not a monitoring system

PmmPack produces a plan. The collectors, dashboards, review meetings, and the person who updates the risk file are still yours. The draft names the signals, the cadence, and the loop, plus the gaps where your setup has nothing behind them. The feedback loop is stated as a decision to be made, not a thing the tool performs. A thin pasted description produces a thin plan, and more signals bring more noise.

The limits, stated plainly

This is a decision-support tool. It is not legal advice, it is not a certification, and it is not a live audit harness. It does not monitor your system, verify that your plan works, or decide whether Article 72 applies to you. That depends on high-risk classification under the Act, including the Annex III use cases, and on your role as a provider. If your system is not high-risk, most of this plan does not apply.

I am referencing real obligations, not a deadline, not a fine, and not a guarantee that a written plan makes anything compliant. Article 72 is about post-market monitoring by providers; Article 9 is the risk-management system it feeds into. The output is a plan starter. It does not run the plan. If your monitoring is a green dashboard and no way to notice drift, the paste-first starter at PmmPack gives you a plan to argue with.

FAQ

What does PmmPack output?

An Article 72 post-market monitoring starter from a description you paste: signal themes, a review cadence, the feedback path into the risk management system, and the evidence gaps in your plan. A draft, not a running monitoring system.

Does it collect signals or run reviews?

No. It has no connection to your dashboards and collects no data. The collectors, meetings, and risk-file updates are yours to build.

Is this legal advice or a certification?

Neither. It is a decision-support draft, not legal advice, not a certification, and not a live audit harness. It cannot verify your plan or decide whether the obligations apply.

Do I need it if my system is not high-risk?

Probably not. Post-market monitoring duties here are tied to high-risk systems, including Annex III categories. If that is not your situation, most of the plan will not apply.