Bayes' Theorem

This article records tradition as it has been passed down and reported. Its sources are not yet part of the atlas's verified catalogue.

Thomas Bayes never published the theorem that carries his name. He was an English Presbyterian minister in Tunbridge Wells who dabbled in mathematics as an amateur interest, well enough to be elected a Fellow of the Royal Society in 1742, but he put out almost nothing under his own name in his lifetime. When he died in 1761, his papers passed to a friend and fellow minister, Richard Price, who found among them an essay working out how to reason backward from an observed effect to the probability of its cause. Price recognized what he was holding, edited it, and in 1763 presented it to the Royal Society as An Essay towards Solving a Problem in the Doctrine of Chances, two years after the man who wrote it had died. The theorem itself answers a question that sounds simple and is not: given how likely some evidence is under a hypothesis, and how likely the hypothesis was thought to be before the evidence arrived, how should that belief in the hypothesis be updated once the evidence is in? Bayes worked out a special case. It was Pierre-Simon Laplace, in 1774, independently and apparently without knowledge of Bayes's essay at all, who derived a substantially more general form of the same idea and did far more than Bayes ever did to develop it into a working method, which is why some historians consider Laplace, not Bayes, the true founder of what is now called Bayesian probability. The name stuck to Bayes regardless, attached by mathematicians writing decades after both men were dead, to an essay its own author never saw into print.

This article records tradition as it has been passed down and reported. Its sources are not yet part of the atlas's verified catalogue.

Most probability, as it was understood before Bayes, ran in one direction: given a known cause, how likely is a given effect? A fair die is known to have six faces, so the probability of rolling a four is known outright. Bayes' theorem runs the calculation the other way, from an observed effect back to the probability of a candidate cause, and that reversal is what has made it, in the centuries since, a working method for medicine, spam filters, courtroom reasoning and machine learning, far outside the doctrine of chances Bayes himself was writing about in the language of eighteenth century probability. In its plainest form the theorem says the probability of a hypothesis, given some observed evidence, depends on three things: how probable the evidence would be if the hypothesis were true, how probable the hypothesis seemed before the evidence arrived, and how probable the evidence was overall, however it came about. A medical test that is genuinely reliable, correctly positive nineteen times out of twenty, can still make a positive result more likely to be a false alarm than a true finding, if the condition it tests for is rare enough in the population being tested. That is not a paradox in the theorem, it is exactly what the theorem predicts once the low starting probability of the rare condition is folded in, and it is the single most common way Bayes' theorem gets invoked to correct an intuition that ignores how rare something was to begin with. The Bayesian approach to statistics generally, the practice of starting from a stated prior belief and updating it in light of new data using this same reasoning, is named for the theorem and remains one of two major competing philosophies of what probability itself means, standing against the frequentist tradition that treats probability only as a long-run frequency.