Discrete Stochastic Processes

Lesson 2 of 10

Markov Chains

A frog that forgets where it has been: one equation per state, and the chain solves itself.

The idea

A process is Markov when the next step depends only on where you are now, not on how you got there. That lets you write one equation per state: the quantity you want (a probability of winning, an expected time) at a state equals the average of its value at the neighbours, plus the cost of one step.