Managing to the Metrics

spolskySince my team runs the operational metrics for my organization, I’ve had some interesting conversations within my team and with some of my peers around the nature of operational metrics, and the uncanny ability for our engineering teams to figure out ways to “game” the system. Not to say that they are abusing the system — it’s just human nature to understand the parameters around which we are being measured and to do everything possible to “optimize” our performance within those boundaries. While reading Joel Spolsky’s book Smart & Gets Things Done about how to find, hire, and retain the best engineering team, one section really jumped out at me. Joel was discussing a book and study conducted by Robert D. Austin from Harvard Business School called Measuring and Managing Performance in Organizations, and stated:

“People are not chemistry experiments, because they are self-aware, and when you try to measure things about them, they’re aware of this, and they have brains they can use to get the measurement to look the way you want it to look.

“(Austin) shows that whenever you institute a new metric in a knowledge organization, that is, any organization with workers who need to do something more complicated than screwing caps on toothpaste tubes, at first you see genuine improvement of the thing you want to measure. The programmers do, actually, try to write more code every day. But very soon what you see is that the workers figure out shortcuts, so that metrics start to go through the roof, while the actual performance actually declines, because programmers start spending more time optimizing for metrics, which comes at the cost of the quality of work that they do.

More importantly, this is not just because you haven’t figured out the perfect metric. It’s the very nature of knowledge work.”

Christian Buckley

Christian is a Microsoft Regional Director and M365 Apps & Services MVP, and an award-winning product marketer and technology evangelist, based in Silicon Slopes (Lehi), Utah. He is the Director of North American Partner Management for leading ISV Rencore (https://rencore.com/), leads content strategy for TekkiGurus, and is an advisor for both revealit.TV and WellnessWits. He hosts the monthly #CollabTalk TweetJam, the weekly #CollabTalk Podcast, and the Microsoft 365 Ask-Me-Anything (#M365AMA) series.

1 Response

  1. danielb says:

    Great book & great quote from Spolsky! In data intensive organizations like (add company name here), there is a tendency to believe that people can be manipulated as easily (and without cost) as numbers on a spreadsheet. Unfortunately, we do not realize that people are much more complex and have a different incentive system- one that can’t be tracked by a database- that often goes against our very best analysis.

    Looking at things from the lateral, or “soft”, side of things as an overlay to the hard numbers is something that can’t be overlooked. In fact, quoting (kind of) from A Beautiful Mind during John Nash’s Nobel Prize Award speech, it is Love, that provides the greatest amount of consistency, certainty, and the true foundation for success and health in our lives- work, personal, and otherwise.