Markov Chain

A Markov chain is a mathematical model for a sequence of possible events in which the probability of each event depends only on the state reached in the previous event, a property known as the Markov property or memorylessness. The chain is named after the Russian mathematician Andrey Markov, who first studied these processes in the early twentieth century, initially applying the idea to the sequence of vowels and consonants in a Russian poem. A Markov chain can be described by a set of possible states together with the transition probabilities of moving from each state to every other state, and when a chain has a finite number of states these transition probabilities can be represented as a matrix. Markov chains are used to model an enormous range of processes, from board games and queueing systems to genetics and the algorithms search engines use to rank web pages.