Constrained Optimization: Lagrange Multipliers

You have 10 units to split between $x$ and $y$ and want to maximize the product $xy$. The constraint $x + y = 10$ lets you eliminate a variable: $y = 10 - x$, so you maximize $10x - x^2$, set the derivative $10 - 2x$ to zero, and get $x = y = 5$ with product 25.

That trick needs a constraint you can cleanly solve for one variable. On the circle $x^2 + y^2 = 5$, solving gives $y = \pm\sqrt{5 - x^2}$: two branches, a square root, and endpoints to check separately. A portfolio with 50 weights, a budget constraint and a return target is worse. You could solve for one weight, but the algebra buries the structure of the problem.

The unconstrained method from Optimization: Gradients, the Hessian and Gradient Descent sets $\nabla f = 0$ and finds the top of the hill. With a constraint, the top of the hill is usually off limits. The best you can do is the highest point along the path you are forced to walk, and at that point $\nabla f$ is typically nonzero. We need a condition that holds at a constrained optimum and treats every variable the same way.