Fifteen Eighty Four

What is Bayesian reasoning and how can we apply it to case studies and qualitative research?

The basic idea is simple—we are social science “detectives,” and our goal is to find any and all information we can to help us figure out the best explanation for whatever we are studying, whether that’s how democratization came about in South Africa, or why South Korea initially managed a more effective response to COVID-19 than many other countries. We use the clues we find in news archives, historical records, interviews, participant observation, or secondary literature to figure out “whodunnit”—namely, which explanation best fits the evidence we uncover.

To put things in Bayesian language, we start by coming up with some rival hypotheses to compare. We assess prior odds—which hypothesis do we initially find more plausible (and to what degree), based on the literature we’ve read and our general understanding of how the world works. We gather evidence, and we evaluate its inferential strength by asking which hypothesis makes that evidence more expected. Here we have to imagine and “mentally inhabit” a hypothetical world where a given hypothesis is true, and ask how well our evidence fits with the kinds of things that would make sense to observe in that world. Then we imagine an alternative hypothetical world where a rival hypothesis is true and do the same. In technical terms, we’re evaluating likelihood ratios for the evidence. Using Bayes rule, we then update our prior odds in light of the evidence to obtain our posterior odds on the hypotheses—Bayes’ rule tells us that that we should gain confidence in whichever hypothesis makes the evidence more expected.

Do you need experience of Bayesian analysis before reading your book?

Not at all! We develop the method from the bottom up. Readers only need to be comfortable with basic arithmetic.

The Bayesian approach itself is quite intuitive and matches well with the way that scholars tend to naturally approach qualitative research, going back and forth between theory revision, data collection, and analysis. At the same time, Bayes gives a rigorous foundation for inference that helps us avoid common cognitive biases of the sort that have been studied by the famous psychologists Daniel Kahneman and Amos Tversky.

What is the most important thing Bayesian reasoning can do for qualitative research?

Just remembering that evidentiary support comes from asking which hypothesis makes the evidence more expected can greatly improve social science. Evidence that is consistent with a hypothesis does not necessarily support that hypothesis—the inferential weight of the evidence comes from the relative likelihood of the evidence under rival hypotheses. We have found many examples where scholars overstate how strongly their evidence supports their argument, because they do not consider whether the evidence might be equally, or even more likely under a rival hypothesis. That’s actually a very common form of confirmation bias that Bayesianism helps us avoid.

What are you working on now?

Our current project applies Bayesian reasoning to reconsider what we know so far about the origins of COVID-19. This is a case with notoriously incomplete information, so it’s a great opportunity for Bayesian analysis—Bayesianism is a prescription for rational reasoning in the face of uncertainty and limited information.
The debate on COVID-19 origins has been politically fraught, but setting aside the most implausible conspiracy theories, there is real disagreement among qualified experts. Some are adamant that the case is closed in favor of zoonosis, but others view the evidence as weak or believe that some sort of lab accident remains a firm possibility.
Using the approach developed in our book, we are assessing how strongly existing evidence favors zoonosis versus lab-leak hypotheses. We are reviewing a broad range of evidence, including epidemiological data, biological data, and sociological information, drawing on published scientific research, journalistic sources, and our own interviews with scientists and area experts. The project highlights the flexibility of Bayesian reasoning—this method can be used to evaluate any kind of evidence, quantitative or qualitative. In fact, Bayesianism forms a bridge between quantitative and qualitative research.
The broader goals of the project include evaluating to what extent a Bayesian framework can help improve reasoning when evidence is both limited and conflicting, communicate degrees of uncertainty more effectively, and pinpoint sources of disagreement among experts on questions with significant public policy implications. To these ends, Bayesianism is a natural framework for handling incomplete information and evaluating uncertainty; it gives results that are easier for laypeople to interpret—namely, posterior odds that tell us how much confidence we should hold in the leading hypothesis relative to rivals; and it provides an excellent template for making our reasoning more systematic and more transparent.