Religious Roulette
🎰 Coin Flips and the Hot Hand Fallacy 🎰
“I’ve been in an endless number of arguments [about the hot hand fallacy]. I’ve won them all, and yet I didn’t convince a soul.” -Amos Tversky
Grab a quarter. I’ll wait.
If you flipped that quarter, what are the odds that the coin will land on heads?
If you’d said 50%, you’re doing great.
Let’s say that you flip the coin four times, and the coin lands on heads all four times. If you flip the coin a fifth time, what are the odds that the coin will land on heads again?
Some of you might see this question and think that since the coin landed on heads four times in a row, we are “due” for the coin to land on tails the next time, and thus we should say it’s less likely that the coin will land on heads than 50%.1 Or perhaps we could go in a different direction, and assume that somehow the coin landing on heads has a “hot streak,” and thus it’s more likely to land on heads the fifth time.
However, the answer is still 50%.
No matter how many times you flip a coin, the chance of landing on heads is equal to the chance of landing on tails. Just because you see a pattern does not mean that a pattern exists.
Or maybe I’m wrong?
Hot Hand Fallacy
In 1985, Thomas Gilovich, Robert Vallone, and Amos Tversky (GVT) published a paper that looked at how people tend to interpret small, random sequences of events as indicative of some larger pattern, what they called the “hot hand fallacy.”2 Their research focuses on basketball players, which is a slightly different case study than coin flipping, as coin flipping is always the same probability, whereas basketball players can theoretically take a shot from an infinite number of spots on the court, and thus each shot has a different probability.
However, any sports fan is familiar with a moment when they thought a player was “hot,” and that she/he/they will obviously make their next shot or hit a home run because they’ve done so on the previous few occasions. Sports fans of all talk about hot and cold steaks all the time.
But what GVT finds is that these streaks do not exist.
Instead, GVT argues that people are inclined to see patterns where none exists, and that “no amount of exposure…will convince the player, the coach, or the fan that the sequences are in fact random.”3 Statistically, “the probability of a…[made shot] is largely independent of the outcome of previous shots,”4 and a player making or missing a few shots in a row is a small sample size and thus statistically insignificant. But people are inclined to argue for the existence of these streaks, in spite of the evidence.
The hot hand fallacy comes up all the time in sports, gambling, politics, and investing. In fact, it is described in detail in this clip featuring Selena Gomez and Richard Thaler in The Big Short:
Below, I am going to mess with your heads about the hot hand fallacy, but consider this question in the meantime: What are areas in Jewish life where we tend to argue that some organization is on a “hot streak,” when no such thing exists? File that answer away for a few moments.
The Hot Hand
As an academic exercise, the hot hand fallacy seems logical. However, try and tell a professional athlete that hot streaks do not exist, and they will dispute this point for hours on end. And turning the world of finance, investment fortunes have been built by people who argue that picking stocks is essentially a useless exercise, and people will make more money in the long term by investing in index funds. As you can imagine, many hedgies take issue with this. Professionals in a variety of fields make their living based on the assumption that momentum and streaks do exist, and as Upton Sinclair once said, “It is difficult to get a man to understand something when his salary depends upon his not understanding it!”
Ben Cohen’s The Hot Hand tells the story of the hot hand from GVT’s paper through the present day. While Cohen also describes how recently two economists named Joshua Miller and Adam Sanjurjo convincingly argued that some version of the hot hand might exist, you can read about that in the footnote.5 What makes Cohen’s book thought-provoking is the range of examples outside of sports where a belief or non-belief in a hot hand has real consequences, sometimes life or death consequences.
In a 2007 paper entitled “Refugee Roulette,” Andrew Schoenholtz, Jaya Ramji-Nogales, and Philip G. Schrag found that if an asylum court judge grants asylum to three people in a row, he/she/they are less likely to grant asylum to the fourth asylum seeker regardless of the merits of the case.6 Cohen describes this case study in great detail and points out how problematic it is that the “asylum court is basically a casino,” where life or death consequences are shaped by a random series of events.7 And no matter where you stand on immigration policy, the idea that life-altering decisions are made based on anything less than the merits is nothing less than a cruel and unusual punishment.
Now, let’s return to my question about the Jewish Community.
In general, most decisions that Jewish organizations make are generally more important than whether or not a basketball player makes their next shot,8 and generally less important than whether or not an asylum seeker goes back to their home country. And if that’s true, think about how many decisions might be based on similarly flawed logic, everything from foundations reading grant requests, camps and day schools deciding tuition assistance, or talking heads opining which Jewish organizations are “hot” or “cold.” No matter the context, we deserve better than flawed logic in our decision-making.
Ben, Talk Nerdy to Me
Giggling…
Weekly Links
The Physics of Kaizen: We are going to come back to kaizen in a future issue, but I loved this piece on why we need to give more credit to people who stopped problems before they became problems.
Stop Brainstorming: A number of the most commonly used leadership buzzwords are profoundly misunderstood. Brainstorming is one of them.
Freelancer Study 2023: Everyone who reads this newsletter relies on freelancers, either because their organization hires them, or they are a freelancer. This is a great study of the state of the field.
What Happens When We Stop Small Talk?: I don’t love small talk and wish I didn’t need to do it. Turns out, that’s probably a mistake. Here’s why.
43 Prompts That Will Change Your Life: I generally avoid listicles, but I am making an exception for this list. The list is practical and thought-provoking.
This is the gambler’s fallacy. We will come back to this another time, but it’s a separate heuristic. But if you’ve ever played roulette, and put down money on “red” or “black,” you’ve probably seen this before. If the dealer spins the wheel four times, and each time the wheel lands on red, one might be tempted to think that the wheel is “due” to land on black the next time. But they would be wrong, and that’s the gambler fallacy. Over time, the wheel will land on red or black in close to equal amounts, but in a small sample size, one could see a pattern where none exists.
Thoms Gilovich, Robert Vallone, and Amos Tversky, “The Hot Hand in Basketball: On the Misperception of Random Sequences,” in Cognitive Psychology, Volume 17 (1985), 295-314.
Ibid., 312.
Ibid.
Joshua B. Miller and Adam Sanjurjo found a flaw in GVT’s research. Miller and Sanjurjo’s research was published in a major economic journal, and I will confess to understanding very little of it. But you can read either this academic piece or an interview with the authors from Scientific American, and see what you think. In essence, what they found is that the original research from GVT was guilty of “selection bias,” when a researcher’s choice about what should be studied actually biases the data, from the start.
For the academic paper, see Joshua B. Miller and Adam Sanjurjo, “Surprised by the Hot Hand Fallacy? A Truth in the Law of Small Number,” in Econometrica, Volume 86, Number 6 (November 2018), 2019-2047.
For the popular version, see Joshua Miller and Adam Sanjurjo, “Momentum Isn’t Magic- Vindicating the Hot Hand with the Mathematics of Streaks,” The Conversation, 28 March 2018.
Once you get past understanding the research (admittedly a tall order), Cohen provides a fantastic account of how Miller and Sanjurjo’s research was viewed by other economists, particularly Daniel Kahneman.
In a climactic moment, Cohen describes the conference where Miller and Sanjuro present their research in front of Daniel Kahneman, whose research partner Amos Tversky produced the foundational research on the hot hand. But in an act that surprised everyone, Kahneman says that the researchers were correct in pointing out a flaw in GVT’s study. At the same time, Kahneman also said that “the point is still valid. People see patterns where there are none.”
And this is another way in which the hot hand fallacy comes up in Jewish communal life. When we see something that is successful, how do we react? And vice versa, how do we react when we see that something is not working? Many trends among Jewish opinion-makers and the funding community are cyclical; certain things are given extra attention because they are “hot,” and certain things cannot get any attention because they are “cold.” But in each case, we must be mindful that human beings tend to see patterns where none exists…
Andrew I. Schoenholtz, Jaya Ramji-Nogales, and Philip G. Schrag, “Refugee Roulette: Disparities in Asylum Adjudication,” in Stanford Law Review, Volume 60 (2007-2008), 295-412.
Ben Cohen, The Hot Hand: The Mystery and Science of Streaks (New York: HarperCollins, 2020), 155.
Other than choosing a captain for Yom Sport or Maccabiah. Now THAT is essential!