Problem 173 of 333MediumCodingP173

Catastrophic cancellation

Computing the sample variance via $\text{Var} = E[X^2] - (E[X])^2$ can fail in floating-point arithmetic for large means and small variance. Why, and what's the fix?
See the hint and solution with a free account
Every problem on Quant Engine is free, with a hint and a full worked solution. Making an account takes 30 seconds and no card. Then you can see them and track what you have solved.
Create a free account Log in

See the hint and solution with a free account