## Hello World, the Lagrange Multiplier Theorem, and Kuhn-Tucker conditionsJuly 26, 2009

Posted by Phi. Isett in Calculus, economics, Uncategorized.
Tags: , , , , , ,

This is a blog which was started on a whim in order to host a very extensive reply to a blog entry of Terence Tao.  This entry is not the reply — the next entry is.

I may continue this blog to contain pieces of mathematics that I find pretty, important and/or not so commonly known.  In any case, I’d feel silly devoting an entire blog to a single reply.  So here.. I’ll take this opportunity to include my favorite proof of the Lagrange multiplier theorem, and later on maybe I’ll post a nice proof of Stirling’s formula, a group algebraic proof of the Riemann-Lebesgue lemma, or something…

The Lagrange Multiplier Theorem helps us solve constrained maximization / minimization problems, making it (among other things) extremely important in economics.  A (weak form of) the Lagrange Multiplier theorem can be stated as follows:

Let $f : \mathbb{R}^n \to \mathbb{R}$ be a real-valued, continuously differentiable function (called the “objective function” — the thing we want to maximize or minimize), and let $g : \mathbb{R}^n \to \mathbb{R}^m$ be continuously differentiable (called the “constraint function” for reasons which will soon be clear).

If $x_0$ is an extremizer of the restriction of $f$ to the (codimension $m$) “constraint manifold” given by $M \equiv \{ x : g(x) = 0 \}$, then there is a unique linear functional $\lambda : \mathbb{R}^m \to \mathbb{R}$ satisfying the equality of linear maps $Df(x_0) = \lambda \circ Dg(x_0)$.

We assume $m \leq n$ — things tend to be more complicated when the number of constraints exceeds the number of degrees of freedom.

Proof:

We know that $f$ must satisfy some first order condition: namely (by further restricting $f$ to trajectories on $M$) that $Df(x_0)v = 0$ for any $v \in \mathbb{R}^n$ which can be realized as a velocity at $x_0$ by a path within the constraint manifold. An exercise in the implicit function theorem shows that these velocities include all of $\mbox{ Ker } Dg(x_0)$ (these are all such velocities — they are the directions in which one can move without changing $g$, and together they are called the “tangent space” of $M$ at $x_0$ under generic circumstances).

Finally, now that $\mbox{ Ker } Dg(x_0) \subseteq \mbox{ Ker } Df(x_0)$, it is a trivial exercise in linear algebra to show that there is a unique linear functional $\lambda : \mathbb{R}^m \to \mathbb{R}$ defined by the identity $Df(x_0) = \lambda \circ Dg(x_0)$.  Basically, we have shown that the zero set of the linear map $Dg(x_0)$ is contained in the zero set of $Df(x_0)$, but because the two maps are linear, their level sets are all translations of the zero level set, and therefore we conclude that every level set of $Dg(x_0)$ is contained in a level set of $Df(x_0)$.

Does anyone know how to turn off the annoying preview thing for hyperlinks..? Never mind, got it.

Edit (29 Aug, 2009):

The same ideas can be used to give a proof of the Kuhn-Tucker conditions for a constrained maximization problem where  we replace the system of equalities $\{ g(x) = 0 \}$ with the system of inequalities $\{ g_i(x) \leq 0 | i =1, \ldots, m \}$.  We simply replace the role of trajectories within the constraint manifold by trajectories going into the constraint set.