Sanov's Theorem gives a large deviations principle for the empirical distribution of independent, identically distributed random variables, stating that the probability the empirical distribution lies close to some other fixed distribution decays exponentially in the sample size, at a rate given by the relative entropy, or Kullback-Leibler divergence, between that fixed distribution and the true underlying distribution. Named for Ivan Sanov, it is a foundational result of large deviations theory quantifying how unlikely a systematic deviation of the sample from the true distribution actually is.
Facts
StatementA bound on the probability of observing an atypical sequence of samples from a given probability distribution. 1 Classification
Statement Form Connections
Has Statement Form
Entity-backed identity for the statement-form enum value this theorem already carries, resolved to a mathematics concept by an explicit value-to-entity map (phase 3 bucket conversion, docs\design_entity_backed_browse_buckets_20260928.md). The statement-form fact itself stays on the theorem unchanged.
In Branch
Source Sanov's theorem (Wikipedia)
Sources
1. Sanov's theorem (Wikipedia)
Introduction
Sanov's theorem gives a bound on the probability of observing an atypical sequence of samples from a given probability distribution.
References
Sanov, I. N. (1957) 'On the probability of large deviations of random variables'.
In Branch: Information Theory, Lead sentence
In mathematics and information theory, Sanov's theorem gives a bound on the probability of observing an atypical sequence of sampl
View the SourceReader Challenges (0)
No disputes yet. Spotted an error or a better source? Open the first one.
Sign in to dispute this or suggest a correction.