In 2004 an intensive care program across roughly a hundred Michigan units drove central-line bloodstream infections to zero and held them there for a decade.
A central line is a tube into a large vein near the heart. An infection in one kills somewhere between one in six and one in four of the people who get it. Zero, sustained, means several thousand people went home who would not otherwise have.
In the same era, New York State published risk-adjusted mortality data for cardiac surgeons. Surgeons began declining the sickest patients rather than risk their scores. The measurement worked exactly as designed: the published numbers improved. It simply moved the risk onto the people least able to carry it, and then made them invisible by removing them from the denominator.
Two measurement programs, same decade, same sector, same instrument in the abstract. One saved thousands of lives. The other quietly rationed care.
The difference is not technique, and it is not the choice of framework. Five conditions separate them, and every one of them is about purpose and ownership rather than method.
1. Professionally owned
Who owns it? The people doing the work chose the measures and use them.
This is the condition that does the most work and the one that sounds most like a platitude, so it is worth calling it out. Ownership removes interference.
When the measure belongs to me, the number is information about my work and I can look at it directly. When it belongs to someone who is watching, the number becomes a verdict about me, and I will spend attention managing the verdict that I could have spent on the work. The same indicator, on the same dashboard, does opposite things depending on whose it is.
Clinicians in the Michigan program could look at their own infection rates without flinching, because nothing followed from the number except learning. Practices under national pay-for-performance schemes learned to document rather than to improve, because for them the number carried a consequence.
2. Embedded in improvement cycles
How often will they look at it? Weekly or monthly, at the front line, not annually at a board.
Cadence is not an administrative detail. A measure reviewed annually can only ever explain what happened. A measure reviewed weekly can change what happens next, which is the only thing that makes it worth collecting. The Michigan units ran checklists and reviewed results continuously; the infection rate was a working instrument rather than a report.
3. Used for learning, not judgment
What happens to them if it is bad? Nothing that touches money, ranking or a career.
The evidence here is unusually consistent and unusually ignored. The UK's Quality and Outcomes Framework, the largest primary-care pay-for-performance scheme ever run, produced modest process gains, no clear effect on mortality, and a cost-effectiveness analysis concluding it was not worth continuing. Readmission penalties in the US reduced readmissions and generated a contested but plausible signal of increased mortality. Performance-based funding in higher education has returned a strikingly consistent null across two decades of study, alongside credential inflation and equity harm.
None of those schemes was badly designed by the standards of their own logic. They were accountability instruments asked to produce improvement, and accountability instruments cannot do that, because the consequence attached to the answer changes what the answer is.
The everyday version, and why it costs more than it saves
What's at play here is easier to see away from the operating theatre.
A call centre sets out to reduce average handling time, so it measures it and holds people to it. Operators start hanging up on callers. Handling time falls, precisely as instructed. Nobody was being difficult: hanging up was simply the cheapest available route to a number they were now being judged on.
Then the callers ring back, because their problem was never solved.
John Seddon named this failure demand after watching call volumes explode when telephone banking moved out of branches in the 1980s. The extra calls weren't new customers. They were the same customers, returning, because nothing had been fixed the first time. His estimate is that failure demand accounts for forty to sixty per cent of activity in a typical call centre, though that figure comes from his own consultancy's work rather than independent study and should be read accordingly.
Follow the math. Average handling time improves. Call volume rises. Total work goes up.
The efficiency gain isn't hollow, it's negative. Which removes the defence usually offered for these schemes, that quality was knowingly traded for cost. There was no saving to trade for. The only thing that reliably increased was the number of calls, and the organisation paid twice: once for the failure, once for the call about the failure.
Seddon's 1992 book on this is called I Want You to Cheat. The title is the argument.
4. Paired with capability-building
Can they act on what it shows? The measure came with the means to act on it.
This is the condition most often skipped, and skipping it is worse than not measuring. A measure that reveals a problem the organisation has no capability to fix produces demoralisation first and gaming second, in that order, reliably.
If the data says handover quality is poor and nobody has the authority, time or training to redesign handover, the only rational response available to the front line is to make the number look better. The measure has not created pressure to improve. It has created pressure to appear improved, which is a different thing and considerably cheaper to supply.
5. Following evidence rather than substituting for it
Does it detect drift, or attempt proof? The model was established elsewhere, and the measure watches whether it is being delivered faithfully.
The Nurse-Family Partnership, an evidence-based home-visiting program, monitors fidelity to its model because randomised trials had already established that the model works. The measurement exists to detect drift from something known to be effective.
Most organisations are asked to do the reverse: to demonstrate, through routine operational indicators, that their model works at all. That is a research question being asked of a reporting system, and reporting systems cannot answer it. They do not have comparison groups, they do not have baselines taken before anyone thought to look, and they cannot rule out the twenty other things happening in a person's life.
Asked to prove the unprovable, organisations produce something that looks like proof. Everyone involved knows what it is.
The technical rule underneath all five
Do not rank.
Ranking noisy, overdispersed data into a league table, which publishes every unit's position against every other, manufactures signal that is not there, and it is the mechanism behind most gaming. Half the units in any league table are in the bottom half by arithmetic, not by failure. Publish the ranking and the units at the bottom will improve their position, which is not the same as improving their care.
Funnel plots with control limits do the job properly: they show which variation is ordinary and which is genuinely exceptional. The practical consequence is one sentence. Act on a point outside the control limits, never on a rank.
What this means for the performance framework you are building
The five conditions are not a maturity model and they do not need sequencing. They are a filter, and they run in one direction: if a proposed measure fails condition three, nothing about conditions one, two, four or five will rescue it.
Which is the whole filter, in the five questions above. Who owns it, how often will they look at it, what happens to them if it is bad, can they act on what it shows, and does it detect drift or attempt proof? Five answers, and most candidate measures fail at least two.
That is a better use of an afternoon than another workshop on indicator design.
This is the evidence behind one section of Is Anyone Better Off?, which sets out why measurement systems report upward and never learn backward. Condition three above is the general case of a narrower, more famous failure: Goodhart's Law is what happens when a single metric, not a whole measurement system, gets used for judgment instead of learning. The related question of whether reporting adds value at all is in nineteen billion dollars without a report.