Lagrange Multiplier

Lagrange Multiplier Visualization

Download URL : Lagrange_Multiplier_Visualizer

Lagrange Multiplier

Local Maxima and Minima

Lagrange multiplier method is used to find local maxima and minima of a function f(x,y) subjected to constraint g(x,y) = k. Let’s explore local maxima & minima of a scalar function before studying lagrange multipliers

Consider a scalar field f(x,y) = x² + y². Now for this function the unique global minimum occurs at x,y = 0 and has no unique global maximum. Now consider a path line segment AB in the scalar field. Local maximum or minimum of f(x,y) is the location in the path AB where the function f(x,y) attains its maximum value or minimum value. See the below picture for reference

Global minimum @ (x,y) =(0,0) but local minimum/maximum is along the line segment AB

Example:

Without using lagrange multiplier we can able to find local maxima/ minima of a function f(x,y) subject to path from A to B if we can able to parameterize the path.

Let f(x,y) = x² + y²

Path AB is a line segment from A(1.43,-0.125) to B(-0.335,-1.17)

x = 1.43t – 0.335(1-t) & y = -0.125t- 1.17(1-t)

Now by substituting x,y we can change f(x,y) to f(t)

f(x,y) = x² + y²

f(t) = (1.43t-0.335(1-t))² + (-0.125t – 1.17(1-t))²

Now the maxima or minima of f(t) can be found by using

Now the local minimum occurs at t = 0.43114267 or at (x,y) = (0.425967,-0.71946)

Concept Behind Lagrange Multiplier

Consider two functions y₁ = 3x and y₂ = x³ as shown in figure below and let us make a third function using y₁ & y₂ such that y₃ = y₁ – y₂ = 3x – x³ seen in the next figure

Function y₃ has maximum/ minimum at x = 1 & -1

Maximum/ Minimum of y₃ can be found by differentiating

So the maximum/ minimum occurs at x = +1 & x= -1 where y₃ =2, -2

Substituting dy3/dx = 0 is also equaling to substituting dy1/dx = dy2/dx

Intuitively we can see that the slope of this two function (y1 & y2) is equal at the maximum & minimum point. See the below figure

At x = 1 & -1

The above is a crude explanation of the principle behind lagrange multipliers

Lagrange Multiplier in Scalar Field

Consider a scalar field f(x,y) = y – x³ and we find the local maximum/ minimum along a straight line AB: y – 3x =0 which is of the form g(x,y) = k. So we have to maximize f(x,y) subject to g(x,y) = k

Lagrange function is defined as

L(x,y,λ) = f(x,y) – λ(g(x,y)-k) where λ is the lagrange multiplier

If f(x,y) and g(x,y) are continuous & differentiable, there exist a stationary point (x0,y0,λ0) where f(x0,y0) is maximum and the partial derivatives ∂L = 0

Now the above statement can be simply written as

And

Let’s solve the problem for more insight

*(-1)

Now solving the 3 equations we get λ = 1 & 9x,y) = (+1,-3) & (-1,+3). Now the maximum & the minimum values of f(x,y) are +4 & -4

See the below figure

Note: Even though the x value is +1,-1 as the 2D plot example above, both the problem are not similar. Here f(x,y) = z = y-x³is a scalar plot in 2D surface but the above problem y = x³is plotted over a line.

If I could explain a bit more on this, the contour lines shown in this plot is drawn for various z values for example if we cut the surface plot with a plane z = k we get those contour lines. Coincidentally the green contour line passing through the origin is for z = 0. The value we are maximizing is along the z-axis pointing through the screen.

Example 1:

Find the local maxima/minima of a function f(x,y) = 3x²+5y² subject to g(x,y) = k which is a straight line of equation ax + by = 1

Solution:

Lagrange function