Discrete Stochastic Processes
Markov Chains for Dummies
A frog that forgets where it has been: one equation per state, and the chain solves itself.
What to remember
- Markov: the future depends only on the present state.
- First-step analysis: one linear equation per state.
- Long run: solve π = πP, or use detailed balance when the chain is reversible.
Read the lessonMarkov Chains, with a checkpoint at the end.