Most incidents are something you shipped

The short version

If you can answer “what changed in the last hour” in seconds, you have solved most of incident investigation. In AI systems the list of things that count as a change is longer than in ordinary software — model weights, quantisation, batch and cache settings, prompt templates, dataset versions — and most teams version only the code. Tag every deployable artefact, record every config change with an author and a timestamp, and make rollback a rehearsed action rather than an improvisation.

01The base rate nobody argues with

Ask any team that has run production for a few years what causes incidents, and the answer is some version of “us, last Thursday”. Hardware does fail and traffic does surprise you, but those are the minority. The bulk of outages follow a deliberate change: a deploy, a config edit, a dependency bump, a flag flipped in a hurry.

Every vendor in the AI operations space builds on this. They call it change intelligence or change correlation, and the pitch is the same: when something breaks, the first thing worth knowing is what moved. It is a sound premise, and you can act on it without buying anything.

The practical version is a question you should be able to answer in seconds, at three in the morning, from your phone: what changed in the last hour, and who did it? If answering that requires cross-referencing three tools and asking in Slack, that is the reliability project, not the monitoring.

02What counts as a change in an AI system

This is where AI infrastructure differs from ordinary software, and where most tracking falls short. The deployable surface is much wider than the repository.

Things that change behaviour, and whether teams usually version them
Change	Typical blast radius	Usually versioned?
Application code	Whatever the service does	Yes
Model weights or adapter	Every output, subtly	Sometimes
Quantisation or precision	Quality and memory, together	Rarely
Batch size, cache size, concurrency	Latency and rejection rate	Rarely
Prompt or system template	Every output, often dramatically	Rarely
Serving stack version	Performance and failure modes	Sometimes
Dataset used for the last fine-tune	Everything, invisibly	Rarely
Driver or CUDA version	Throughput, or nothing running at all	Rarely

The rows marked rarely are where the difficult incidents live. A prompt template edited directly in a dashboard leaves no trace in git, changes every response, and is nobody’s idea of a deploy — but it is a production change with a wide blast radius. The same is true of a batch-size tweak made to relieve pressure during a busy hour and never reverted.

03Correlating change with behaviour

Once changes are recorded, correlation is mostly bookkeeping. Two mechanics do most of the work.

Annotate your graphs. Every dashboard that shows latency, error rate or throughput should show vertical markers for deploys and config changes. This is a small amount of plumbing and it converts “latency rose at some point” into “latency rose here, and here is what happened here” without anyone reasoning at all. Most monitoring stacks support annotations and most teams never wire them up.

Compare against the same window before the change. Not against an absolute threshold. Traffic has daily and weekly shape, so “error rate is 0.4%” means nothing without knowing it was 0.05% at the same hour last week. Change detection is a comparison, not a limit.

The subtlety in AI systems is that the interesting regressions are often not in the operational metrics at all. A model update that makes answers worse but not slower will sail past every latency and error dashboard you own. That needs an output-quality check running continuously against a fixed set of cases — the same evaluation set you built before shipping, run on a schedule rather than once.

04The model version problem

“Which model is in production right now” should be a trivial question. It frequently is not, for a few recurring reasons.

Tags get reused. Something pinned to latest or to a mutable tag changes underneath you with no deploy event at all — the classic case where nothing changed and yet everything did. Pin to an immutable digest, and treat moving that pin as a deploy.

Adapters compose. If you serve a base model with a LoRA adapter on top, there are two artefacts and either can move independently. Both need versions, and the pair needs recording together.

Quantised copies drift from their source. A 4-bit build of a model is a distinct artefact with distinct behaviour, and “same model, different precision” is not the same model for debugging purposes.

The fix is unglamorous: emit the exact identity of everything loaded — base model digest, adapter digest, quantisation, serving stack version, key runtime settings — into your logs at startup, and expose it on a health endpoint. Then “what is running” is a request rather than an investigation. It takes an afternoon and it pays for itself the first time you need it.

05Making changes attributable

Attribution is not about blame; it is about knowing who to ask. Three habits cover most of it.

One change log, all change types. Deploys, config edits, model promotions, infrastructure changes and manual interventions in one ordered stream with actor and timestamp. It does not need to be sophisticated. It needs to be complete, because a change log missing the category that caused your incident is worse than none, having taught you to trust it.
No untracked production edits. If a value can be changed in a UI, that UI should write to the change log. If it cannot be made to, the value should not be changeable there.
Small, frequent, labelled. A release containing one change is trivially attributable. One containing thirty is a bisection exercise while users are affected.

06Rollback is a feature you build, not a button you hope for

Knowing what changed is only useful if you can undo it, and undo is harder in AI systems than in stateless web services.

Reverting to a previous model means the previous weights are still on disk, or still pullable, and that loading them does not take twenty minutes — which it will if they come over the network on a cold start. Keeping the last known-good build on local storage turns a twenty-minute recovery into a two-minute one, and disk is cheap next to the outage.

Some changes are not reversible in the obvious way. A fine-tune that has already written outputs into a downstream store, or a prompt change that altered records users have acted on, leaves consequences that a rollback does not undo. Those are worth identifying in advance, because they are the changes that deserve a staged rollout rather than a flip.

And rollback should be rehearsed. A procedure nobody has executed is a hypothesis. Run it deliberately, in daylight, before you need it at 3am — the same argument as testing backups.