# Is the Lagrange function objective plus lambda times constraints or objective minus lambda times constraints?

My question is: plus or minus?

Is the Lagrange function:

• the objective PLUS lambda times constraints, or
• the objective function MINUS lambda times constraints?

Example: want to maximize A=xy subject to g(x,y)=2x+y-400=0

is F(x,y,lambda):

• xy + lambda (2x+y-400), or
• xy - lambda (2x+y-400)

I found both notations. Does that mean one can use them interchangeably (i.e. they are the same)?

Thanks for help

-
The tag "machine learning" doesn't seem very appropriate there. Maybe "optimization" or something like that? –  chl Nov 16 '10 at 8:31
I agree, but as I am new here I couldn't create that tag. –  another_day Nov 16 '10 at 8:36
they are not the same but if you minimize them with respect to lambda in the whole real line changing lambda in -lambda does not change the solution. Not that there is a mathematic stackoverflow somewhere (This site is more for statistics) –  robin girard Nov 16 '10 at 8:54

It is exactly the same!
You want the constraint to be respected, and you don't care about the sign of g(x,y)

-
But indeed I am not sure stats.stackexchange.com is the best place to ask your question –  RockScience Nov 16 '10 at 9:27
.. and it is not the best place to comment the question ;) –  robin girard Nov 16 '10 at 9:54
Thus it does not even play a role whether I want to maximize or minimize the function A? –  another_day Nov 16 '10 at 11:07

If you have an optimization problem of the form:

$$min_{x} f(x) \\ s.t. \\ g(x) = 0 \\ h(x) \leq 0 \\$$

Then the Lagrangian is

$$L(x,\lambda,\nu) = f(x) + \lambda g(x) + \nu h(x)$$

with $\lambda \in \mathbb{R}$ and $\nu \geq 0$.

The original optimization problem is equivalent then to $$\min_{x} \max_{\lambda \in \mathbb{R}, \nu \geq 0} L(x,\lambda,\nu)$$.

Why? Because for any point $x_0$ with $g(x_0) \neq 0$, then the inner maximization will cause the term $\lambda g(x_0)$ to "blow up." The same is true for any $x_0$ with $h(x_0) \geq 0$.

So, long story short, for exact equality constraints, the langrange multiplier is a real number, so it doesn't matter if you do plus or minus.

For inequality constraints of the form $h(x) \leq 0$, do plus.

See Boyd's notes on duality for more background: http://www.stanford.edu/class/ee364a/lectures/duality.pdf

-