This Article by Andrew Gelman is one of the very few I’ve read that gets it right: we should be combining Falsification with Bayesian methods.
I’ve called it Bayesian Falsification, he calls it Falsificationist Bayesian, but the point is the same.
The classical or frequentist approach to statistics (in which inference is centered on significance testing), is associated with a philosophy in which science is deductive and fol-lows Popper’s doctrine of falsification. In contrast, Bayesian inference is commonly associated with inductive reasoning and the idea that a model can be dethroned by a competing model but can never be directly falsified by a significance test. The purpose of thisarticle is to break these associations, which I think are incorrect and have been detrimen-tal to statistical practice, in that they have steered falsificationists away from the veryuseful tools of Bayesian inference and have discouraged Bayesians from checking the fitof their models. From my experience using and developing Bayesian methods in socialand environmental science, I have found model checking and falsification to be central inthe modeling process.
“Popper has argued (convincingly, in my opinion) that scientific inference is not inductive but deductive…”
On what he calls the Standard View of the Philosophy of Statistics which stuffs everyone into two competing camps:
The point of this article is that [the Standard View of the Philosophy of Statistics] is a bad idea and that one can be a better statistician—and a better philosopher—by picking and choosing among the two columns rather than simply choosing one.
For example, there are objective Bayesians who feel comfortable choosing prior distributions via the same sorts of theory-based principles that non-Bayesians have commonly used when choosing likelihoods. And there are concepts andmethods such as robustness, exploratory data analysis, and nonparametrics which do not traditionally fall in one or the other column. Overall, though, [the Standard View] represents a set of associations that are commonly held—and I argue that they are influential, and can be harmful, even to those practically-minded folk who have no interest in foundations or philosophy. The economist (and Bayesian) John Maynard Keynes famously remarked, “even the most practical man of affairs is usually in the thrall of the ideas of some long-dead economist”, and this is the case for philosophy as well.
You had me at “rejected” 🙂 …
I follow Popper in believing that a model can be rejected, never accepted. I will go even further and say that, realistically, all my models are wrong. The encouraging message of the present article is that we can have all the powerful data-analytic tools of Bayesian inference, and falsification too!
I’d take this further and argue it’s the very basis of an honest overall philosophy of science, not just stats.
Now go lift something heavy,
PS. In the next issue of Nemesis, I’ll be discussing the problem of induction and Popper’s solution to it.