False Confidence Theorem

The false confidence theorem, published by Balch, Martin and Ferson in a 2019 paper in Proceedings of the Royal Society A, is a result in statistical inference showing that an analyst using an ordinary, additive probability measure to represent uncertainty can be led to assign high confidence to a false assertion whenever the underlying data are of sufficiently low quality. Because additive probability measures require belief in complementary outcomes to sum to a fixed total, weak evidence against one possibility can force belief in an alternative upward even though the data do not actually support that alternative; the theorem uses the example of predicting whether two orbiting satellites will collide to illustrate the danger. The result motivates the use of non-additive alternatives to ordinary probability, including inferential models, confidence distributions and confidence boxes, evaluated for reliability against a standard called the Martin-Liu validity criterion.