What Are The Metrics?
🫤 And other fancy-sounding questions... 🫤
“In situations where there are no real feasible solutions to a problem, the gathering and publication of performance data serves as a form of virtue signaling. There is no real progress to show, but the effort demonstrated in gathering and publicizing the data satisfies a sense of moral earnestness. In lieu of real progress, the progress of measurement becomes a simulacrum of success.” -Jerry Z. Muller
When a person says “metrics” too often, I get nervous.1
(Yes, I contain multitudes.)
But Moneyball Judaism is about a way of thinking, not a longing for a single number to solve our problems. When any word is blindly used, it becomes akin to idol worship.
Furthermore, the word “metrics” can be used reflexively to confer a patina2 of sophistication; it sounds like a person is impressive when they ask, “What are the metrics?” However, asking about metrics is not impactful; knowing how to identify and use them effectively is. And metrics can be misused and manipulated when not adequately understood.
Each reader will need to decide whether or not their organization falls into some common traps, but this week, I will focus on two. Buckle up, friends.
Goodhart’s Law
I don’t envy homicide detectives.
The Wire is one of my favorite TV shows, and it’s clear that creator David Simon wants people to see how hard it is for homicide detectives to change the system, particularly how crime statistics can be manipulated. Consider:
In a city, a key indicator of crime is how many murders occur each year and the percentage of those murders solved by police.
At first glance, this appears to be a reasonable metric. After all, who would argue that it's not better to have fewer murders each year?
But let’s say one December, a detective suddenly discovers that there is a killer who murdered at least one person over the past year and, likely, many, many more.
When this detective tells their superiors, the superiors order the detective not to look for any bodies until the new calendar year—if the detective finds more bodies now, it will kill the yearly clearance rate.
This scenario is a major plotline of season 4 of The Wire:
This episode perfectly exemplifies this week’s big idea, Goodhart’s Law.
Goodhart’s law states, “When a measure becomes a target, it ceases to be a good measure.”3 Yes, it’s great to have fewer murders. However, the managers in this situation are not reducing the number of murders; they are manipulating how investigators use their time so that it will seem like there are fewer murders. The metric is worshipped to the point that it loses utility.
In what will likely only interest people who are as obsessed with these concepts as I am, who exactly “discovered” Goodhart’s law is complicated:
Goodhart’s Law is named after the economist Charles Goodhart, who wrote in a 1975 paper on British monetary policy that “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”4
However, there is some question as to whether or not Goodhart is the first person to identify Goodhart’s law, as a similar principle is identified in a 1976 paper by Donald Campbell5 on how social science data can be manipulated. Campbell writes, “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”6
And to add a layer of complexity, the standard phrasing of Goodhart’s Law that “When a measure becomes a target, it ceases to be a good measure” comes from a 1997 article by anthropologist Marilyn Strathern on British educational policy.7
For our purposes, Strathern’s analysis is most relevant because she examines how too much focus on metrics can lead to a “new morality of attainment.” Strathern argues that once we assume that “human performance could be measured,” we start to think that what we can measure ought to be measured. And because some things can be measured, and others cannot, that which can be measured becomes “enticing tools for improvement,” leading people to spend more time aiming to maximize the specific metric instead of the greater goal the metric should indicate.8
Returning to our serial killer, the only reason a police department counts homicides is to decrease the number of homicides over time. However, once that metric becomes an incentive, where their yearly clearance rate judges police, the measure becomes less valuable because there is an incentive to manipulate it.
Turning to the Jewish organizational world, this is one of the issues with worshipping at the altar of metrics. To put it another way:
The goal isn’t synagogue membership; the goal is Judaism.
The goal isn’t Shabbat meals; the goal is Shabbat.
The goal isn’t trips to Israel; the goal is Israel engagement.
And so on. Yes, there is value in measuring engagement, interactions, connections, etc., but we need to be careful lest people focus on the metric to the point where the mission suffers. And it happens more than we think.
Which brings us to McKinsey.
When McKinsey Comes to Town
While internal leaders can worship metrics, organizations can also be encouraged to worship specific metrics through outside consultants.
The Jewish organizational world has an enormous array of private consultants, coaches, and search firms that play a growing significance in the decisions of many organizations. As far as I know, we know very little about the state of the field despite its size and scope; we will return to this topic in future issues. While the book is not an apples-to-apples comparison, if you are curious about the power and perils of consulting, consider reading When McKinsey Comes to Town by Michael Forsythe and Walt Bogdanich.
F&B’s account of McKinsey examines a global consulting firm that does not make products but impacts countless businesses, governments, and nonprofits. Additionally, evoking “McKinsey” in conversation is another way to provide a patina of sophistication in a leadership role.
To say F&B’s investigation is not flattering towards McKinsey is a gross understatement. However, F&B’s primary concern is not that McKinsey is ineffective or a waste of money, but instead that McKinsey has been effective in problematic industries while insulating themselves from the impacts, a two-step McKinsey has pulled on issues including opioid addiction,9 global dictatorships,10 ICE,11 and pollution.12
If I had to choose the most significant problem about McKinsey and other global consulting firms identified by F&B, it would be how McKinsey plays a substantial role in some of the most problematic decisions made by companies and governments over the past several decades while insulating themselves from the results. This is particularly harmful in two ways:
Moral Agency: By taking this stance, McKinsey argues they have no moral agency when advising clients. This came up in an exchange about McKinsey’s contract on ICE’s deportation policies, where the executive told young employees, “We don’t do policy,” he said. “We do execution,” only for one brave employee to speak up and argue that “With that logic,...you could justify working for any despot, even the Nazis.”13
Sugarcoating Reality: In an interview with John J. Lawler, who studies industrial relations and is critical of the economic impact of management consultants, Lawler argues that “Clients like to be told they are doing the right thing,”14 and bringing in a consultant allows clients to justify the employment of harmful practices by claiming they brought in an “objective” voice from the outside. To put it another way, how objective can one be when they receive a salary from the client?
Perhaps F&B’s analysis is unfair to McKinsey, as there are other firms with similar practices (e.g., Bain, BCG, etc.), and McKinsey also provides research on societal goods. But since I suspect many organizations start talking more about metrics because an executive or outside consultant encouraged them to, it’s worth reading this cautionary tale and deciding how much it applies.
Carnegie Council Podcasts
20
Research from Dr. Alberto Minetti, a physiologist and biomechanist at the University of Milan, shows that a person burns 20 times more calories walking up stairs than walking on flat ground.
What I Read This Week
Campuses Are Ground-Zero on Debates about Anti-Semitism: I rarely read hot takes on anti-Semitism on campus, mainly because so many authors have ulterior motives. However, this history by Jonathan Krasner was fair and well-researched and should be read by anyone.
Heteropessimism and the 4B Movement: I will be curious to see how much this idea gains steam in the coming years. And while not editorializing too much, if I were a woman, I’d be furious, too.
How One Woman Became a Scapegoat for a Reading Crisis: I did not know that there was a reading crisis in the United States. This article was a fascinating introduction and a reminder that fetishizing any one idea does not always have a happy ending.
The Influence of Bell Labs: I haven’t written much about Bell Labs, but it’s hard to understate the role of this research facility in numerous innovation sectors. So read up, and ask yourself what it will take for the Jewish World to create its own Bell Labs.
My Camp: This is the first time I’ve recommended a piece of fiction in this newsletter. However, I love Joshua Cohen’s writing; this piece speaks to our moment in many ways.
I suppose I could say the same thing for “KPIs.”
Kind of like using the word “patina” in a newsletter.
Some people call it “Campbell’s Law”.
Marilyn Strathern, 308.
Ibid., 307.