Judaism’s Kristol Ball
🔮 Maybe a Monkey Could Do This? 🔮
“Those who have knowledge, don't predict. Those who predict, don't have knowledge.” -Lao Tzu
Have you ever gazed into a Kristol Ball?
Nope, not a typo.
In the world of political punditry, “Kristol Ball” refers to the political predictions of Bill Kristol. It is not a compliment. Although Kristol is often asked to opine about future political events, a consensus exists that many of his political predictions are completely wrong. If anything, if Kristol predicts something, many argue that this is all the proof one needs that the opposite will take place. Ouch.
I have sympathy for Bill Kristol; our media ecosystem rewards people who make loud predictions, no matter the consistency of their wrongress (I’m looking at you Skip Bayless and Stephen A. Smith).1 If the public were not consistently willing to listen to prognosticating pundits whose accuracy is suspect, those pundits would be out of a job quickly, or start making better predictions.
But while politics and sports are easy targets for pundits who make bad predictions with no consequence, I’m not sure if the Jewish world is any better.
Remember when Orthodox Judaism was going to disappear? Or when independent minyanim were going to replace synagogues? Or when American Judaism was going to disappear due to rampant secularism?
At some point, this claim was made in a number of public forums by respected “thought leaders” (goodness, I hate that word). But none of these predictions came true, and, in most cases, the opposite took place:
Orthodox Judaism is doing just fine, numbers-wise (to put it mildly)2
Synagogues may be different, but the alternatives are no more stable
Of course, each of these trends is more complicated than a soundbite, but that’s the point. We are willing to give a forum to the hot take, but the certainty with which people made these predictions was much louder than the deafening silence that came when the hot take turned cold.
Frankly, it seems like pundits are always willing to make new wrong predictions, and we are always willing to listen to them (and often believe them).
The question is why…
Repetition Bias
I loathe vaccine denial.
In general, I think it’s important to tolerate a wide range of beliefs. And while I don’t always maximize my tolerance in practice, I get why I should.
I draw the line at vaccine denialism because not having a vaccine can quickly be the difference between life and death. And since we receive many of our first vaccines when we are minor children, I particularly loathe parents who pretend that they know better than medical science, since their children are too young to understand the implications of their parents’ mistakes. Your child should not die because you chose to believe something false on the dark web.
Neuroscientist Tali Sharot is in my camp and wants people to get vaccinated. But she understands far better than I do something called the “repetition bias,” the predisposition of people to believe that which they have been told most often.3 The repetition bias relates to something else called the illusory truth effect,4 the idea that somehow repeated information is “more true” because it is repeated.
As a result, the repetition bias dampens the effectiveness of our natural instincts for fighting vaccine denial. In The Influential Mind, Sharot argues that while our first instinct to fighting vaccine denial is providing information to bust myths about vaccines, Sharot argues that “by repeating the myths regarding…[vaccines] in an attempt to dispel them, people sometimes wind up remembering the myths rather than the counterevidence.”5 Sadly, by giving too much of a forum to vaccine denialists, the lies become self-justifying.
Instead, Sharot outlines how a group of researchers found that they could convince parents skeptical of vaccines to vaccinate their children by “implanting”6 a new belief that led to the desired result, instead of trying to change the parents’ initial vaccine skepticism. She writes:
“Our instinct is to try to alter people’s beliefs and actions by introducing data to prove that we are right and they are wrong…Instead, find arguments that rely on common ground. For example, telling parents who refuse to vaccinate their children that science has shown that vaccines do not cause autism did not alter the parents’ behavior. Instead, saying that vaccines would protect their children from deadly diseases was more effective—the argument did not contradict their prior beliefs and was compatible with the common goal of keeping children healthy.”7
Sharot’s case study demonstrates the power and pitfalls of repetition. Vaccine skeptics have been allowed for too long to share nonsense in countless forums; we cannot un-ring the bell and stop them from saying things that are false.8 Instead, we fight the repetition bias by bringing a new idea into the picture, one that might be able to win out the more we repeat it.
Returning to Jewish life, while it irritates me to no end that certain false predictions are repeated in the face of years upon years of evidence to the contrary, I need to remember that telling someone that their prediction is wrong is a terrible strategy. The repetition bias is too powerful. Instead, I need a better set of tools to encourage more accurate predictions about the future.
Superforecasting
As you’ve probably guessed, I’m not a fan of people who confidently make predictions based on little evidence.9 But my pre-disposition runs into a challenge: If we tend to associate strong leadership with confidence and willingness to make tough decisions, how do we reconcile that ideal with the possibility that the best predictions are made by those who acknowledge uncertainty?
This is where Phil Tetlock comes in.
Tetlock studies how well people are able to predict future real-world events. Spoiler: In Expert Political Judgment, Tetlock finds that political scientists across the political spectrum struggle to perform better at forecasting world events than “dart-throwing chimps.”10 Just wait until the monkeys write Shakespeare:
Of course, the Jewish community has no mechanism for rating predictions, so we lack the opportunity to have a data-based conversation on who makes the most accurate predictions. However, I've written about Tetlock before, and I think it’s reasonable that if someone of Tetlock’s stature finds that experts in political science do not predict real-world events with great accuracy, it’s reasonable to assume that most Jewish thought leaders are no better.11
But Tetlock is not just a critic; instead, Tetlock and Dan Gardner write Superforecasting: The Art and Science of Prediction to outline what we can learn from those people who are successful at predicting future events, and how to use the intellectual humility possessed by most superforecasters in our work. In the appendix, they list ten commandments aspiring superforecasters should follow:12
Triage
Break seemingly intractable problems into tractable sub-problems
Strike the right balance between inside and outside views
Strike the right balance between under- and overreacting to evidence
Look for clashing causal forces at work in each problem
Strive to distinguish as many degrees of doubt as the problem permits but no more
Strike the right balance between under- and overconfidence, between prudence and decisiveness
Look for the efforts behind your mistakes but beware of rearview mirror hindsight bias
Bring out the best in others and let others bring out the best in you
Master the error-balancing bicycle
Don’t treat commandments as commandments
As you can imagine, the eleventh commandment is my favorite =). But seriously, if I had to choose one action that threads throughout these commandments, it would be the importance of calibration.
People who make good predictions know how to calibrate their level of certainty, hear opposing views, and show honesty in the face of mistakes. The above commandments use words like “strive” or “strike” because most of them involve holding two alternative ideals at the same time. This may be difficult, but no one ever said that it would be easy.
Julia Galef and Phil Tetlock
In case you’ve missed hearing from Dr. Galef since she was featured.
Weekly Links
Barbie’s Grand Ambitions: I doubt I will see the new Barbie movie starring Margot Robbie and Ryan Gosling, and directed by Greta Gerwig. But I think that the conversation around the movie has been fascinating; read this piece from The New Yorker and this piece in The New York Times Magazine.
What’s Your Desk Situation?: I kid you not, but there is a newsletter that only contains photos of different ways to organize your desk. And yes, I’m a subscriber.
Fill the Pews By Welcoming Everyone In: I’m naive, and happen to think that any synagogue can be successful depending on the tactics one employs. Here’s more proof (though I could still be wrong).
How To Be An Email Ninja: More articles on how to conquer your inbox! This article is by subscriber Jeremy Caplan. One of the better articles I’ve read on the subject.
The Pandemic Hurt Teens- Here’s How We Can Help: I still think about Jewish teenagers constantly, and it’s a mistake for any of us to forget that adolescents are living in a difficult time. Be part of the solution.
Personally, I think we should have a sports channel where all airtime is filled by Skip Bayless, Stephen A. Smith, Colin Cowherd, Max Kellerman, Jim Rome, etc. Then I know exactly what channel not to watch at all times.
Let’s take a prediction closer to home…
I’m a Conservative Rabbi. I have been told my entire adult life that Conservative Judaism (CJ) is going to die. If you say that the first time I heard this was when I was 20 years old, I’ve been hearing this claim for almost 20 years. The best I can tell, CJ is still alive. And even if CJ dies in the next five years (still unlikely), does making a claim twenty-five years ago and it eventually coming true mean one of these pundits was “right”? I doubt it.
While I could not find a single person who discovered the idea of a repetition bias, I am going to share one example of how to overcome it using Sharot’s work.
Aumyo Hassan and Sarah J. Barber, “The effects of repetition frequency on the illusory truth effect,” Cognitive Research: Principles and Implications, Volume 6, Issue 38 (2021).
Tali Sharot, The Influential Mind: What the Brain Reveals About Our Power to Change Others (New York: Henry Holt & Co., 2017), 30.
Ibid., 30-31.
Ibid., 32.
I won’t name the people who repeat these lies or the lies they repeat because that’s part of the problem. They do not deserve the airtime. Again, get yourself vaccinated.
Truthfully, there is strong evidence to suggest that confidence, or at least over-confidence, is one of the hallmarks of bad leadership. See Don A. Moore and Max H. Bazerman, “Leadership and Overconfidence,” Behavioral Science and Policy, Volume 8, Issue 2 (2022), 59-69.
Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton: Princeton University Press, 2006)
Frankly, given the fact that political scientists have the benefit of at least some data, and many of their predictions can reach binary conclusions (e.g. did this politician win or lose an election?), I think a good case could be made that most people who make predictions about the future of Judaism are far, far less accurate than the political scientists studied by Tetlock.
Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction (New York: Crown, 2015), 277-284.