Problem bank
Problem 321 of 333HardCalculusP321
Choosing a learning rate for a narrow valley
You minimize by gradient descent with a fixed step size : .
- For which does the method converge from every starting point?
- Which gives the fastest guaranteed convergence, and how many steps does it need to cut the distance to the minimum by a factor of ?
- A colleague plays safe with . How many steps does that take in the worst case?
- How many steps would Newton's method need?
See the hint and solution with a free account
Every problem on Quant Engine is free, with a hint and a full worked solution. Making an account takes 30 seconds and no card. Then you can see them and track what you have solved.