The idea
A process is Markov when the next step depends only on where you are now, not on how you got there. That lets you write one equation per state: the quantity you want (a probability of winning, an expected time) at a state equals the average of its value at the neighbours, plus the cost of one step.