Jensen-Shannon Divergence

The Jensen-Shannon divergence is a method of measuring how similar two probability distributions are, named after Johan Jensen and Claude Shannon and also called the information radius. It is built from the Kullback-Leibler divergence but improves on it by being symmetric between the two distributions and always finite, computed as half the divergence of one distribution from the average of the two plus half the divergence of the other distribution from that same average. Smaller values indicate more similar distributions, and the square root of the Jensen-Shannon divergence is itself a proper metric, called the Jensen-Shannon distance. It is used in fields such as bioinformatics and machine learning wherever probability distributions need to be compared.