Sanov's Theorem gives a large deviations principle for the empirical distribution of independent, identically distributed random variables, stating that the probability the empirical distribution lies close to some other fixed distribution decays exponentially in the sample size, at a rate given by the relative entropy, or Kullback-Leibler divergence, between that fixed distribution and the true underlying distribution. Named for Ivan Sanov, it is a foundational result of large deviations theory quantifying how unlikely a systematic deviation of the sample from the true distribution actually is.
Facts
StatementA bound on the probability of observing an atypical sequence of samples from a given probability distribution. 1 Classification
Statement Form Connections
Has Statement Form
In Branch
Source Sanov's theorem (Wikipedia)
Sources
1. Sanov's theorem (Wikipedia)
Introduction
Sanov's theorem gives a bound on the probability of observing an atypical sequence of samples from a given probability distribution.
References
Sanov, I. N. (1957) 'On the probability of large deviations of random variables'.
In Branch: Information Theory, Lead sentence
In mathematics and information theory, Sanov's theorem gives a bound on the probability of observing an atypical sequence of sampl
In Group: Probability Theory, Lead section
from a given probability distribution. In the language of large deviations theory, Sanov's theorem identifies the rate function
View the SourceReader Challenges (0)
No disputes yet. Spotted an error or a better source? Open the first one.
Sign in to dispute this or suggest a correction.