Dukenfield’s Law of Incentive Management

Mark Kleiman, professor of public policy at the UCLA School of Public Affairs and editor of the Journal of Drug Policy Analysis, is a temporary guest-blogger for Ta-Nehisi Coates of The Atlantic Monthly.  He wrote a post on Friday about scandals in which educators were caught falsifying results of tests used to measure school performance and, in the process, came up with a new sociological “law.”

Mark Kleiman

A school superintendent allowing his staff to doctor students’ answers on a set of high-stakes standardized exams has something in common with a corporate CEO holding a bundle of stock options who practices “earnings management” via bogus asset sales. Each is responding to an intense incentive system by faking success rather than producing it.

One could formulate this as a general principle: any incentive to create a result also creates an incentive to simulate the same result. The corollary is obvious: the greater the incentive, the greater the temptation. Or, as W. C. Fields put it in You Can’t Cheat an Honest Man, “If a thing is worth winning, it’s worth cheating for.” Borrowing Fields’s real name, I propose to call this generalization Dukenfield’s Law of Incentive Management. Designers of control systems ignore Dukenfield’s Law at their peril, and ours.

A second corollary follows directly from the first: holding the level of audit effort constant and other things equal, the reliability of a measure will decline as the importance attached to it grows. To put the same thing another way: to maintain a given level of reliability, the resources invested in verifying any performance measure need to rise roughly in proportion to the stakes involved

via The Atlantic. (my bold-facing)

Kleiman wrote that there are other ways to manipulate test results besides obvious cheating. You can encourage the weaker students to transfer to other schools. If the tests measure literacy, you simply eliminate art and music, and cut back on science, history and all other types of instruction except basic reading and math.  You thereby defeat the purpose of the test, which is to gauge educational performance overall.

… Since all incentive systems generate results-simulation, and more powerful incentive systems generate results-simulation more powerfully, counter-simulation strategy should be central, rather than peripheral, to incentive-system design. In general, we should expect the costs of monitoring to rise along with the stakes created by the incentive system.

It turns out that the maxim “If you can’t measure it, you can’t manage it” expresses only half the truth. To manage one must be able, not only to measure, but to measure in the face of active impression management among those measured. Any proposal for an incentive system without explicit consideration of the simulation problem and how to deal with it should be presumed non-serious. If it’s true that management without accountablity is just cheerleading, it’s also true that creating big incentives without preventing results-simulation is just asking to be cheated.

via The Atlantic.


Kleiman’s article reminds me of a saying of W. Edwards Deming, father of Total Quality Management:  “If you give a manager a quantifiable goal, he’ll achieve it if he has to destroy the company in the process.”

Click on The Reality-Based Community for Mark Kleiman’s own web log.

Ta-Nehisi Coates is a senior editor of The Atlantic Monthly.  Click on Ta-Nehisi Coates for his web log.

Tags: , , , , ,

One Response to “Dukenfield’s Law of Incentive Management”

  1. Can we grade the teachers? « Phil Ebersole's Blog Says:

    […] on Dukenfield’s Law of Incentive Management for the pitfalls of linking financial incentives to test […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.


%d bloggers like this: