﻿ Pontryagin's Maximum Principle
San José State University

applet-magic.com
Thayer Watkins
Silicon Valley
USA Lev Pontryagin

 Pontryagin's Maximum Principle

Pontryagin's Maximum Principle applies to a particular type of problem called a Bolzano Problem. Most optimization problems can be put into the form of a Bolzano problem, but more about that later.

A Bolzano problem involves a number of state variables which can change over time where time t runs from 0 to T. Let us suppose the state variables are X1(t), X2(t), ..., Xn(t). We want to maximize

#### V(T) = c1X1(T) + c2X2(T) + ...+cnXn(T),

given that we start at the point X1(0), X2(0), ..., Xn(0), and where the coefficients c1, c2, ..., cn are given and T is some definite finite time. We are given so-called steering functions for controlling the changes in the state variables; i.e.,

#### dX1/dt = f1(X1, X2, .., Xn, u1, u2, .., um)dX2/dt = f2(X1, X2, .., Xn, u1, u2, .., um) ........................................................... dXn/dt = fn(X1, X2, .., Xn, u1, u2, .., um)

where the variables u1, u2, ..., um are functions of time and are called the control variables. The objective is to choose the control variables at each instant of time so as to steer the state variables from their initial values

to some point

#### X1(T), X2(T), ..., Xn(T)

where V(T) = c1X1(T) + c2X2(T) + ...cnXn(T) is maximized.

This seems to be a very difficult task. Pontryagin's Maximum Principle provides a neat, systematic solution.

To implement Pontryagin's method one defines a Hamiltonian function

#### H = φ1f1 + φ2f2 + ... + φnfn = Σφifi,

where the set of adjoint variables φ1, φ2, .., φn are such that

#### dφj/dt = −∂H/∂Xj= −Σi φi(∂fi/∂Xj)

and φi(T)= ci for i=1, 2, .., n. Note that if H does not depend upon Xj then dφj/dt=0 for all t and thus φjj would be said to be conserved.

The optimum value of the control variables at time t are the ones that maximizes H.

This usually means that the optimum uk(t) is such that

#### ∂H/∂uk(t) = 0 which means ΣiXi(∂fi/∂uk(t)) = 0 for k=1, ..., m.

unless uk is constrained, in which case the optimal uk may be at a limit of its range.

## An Example of a Problem of Bolzano in Economics:

Suppose an individual has a noninterest income of y(t) for 0≤t≤T which can either be consumed or saved at an interest rate of r. The individual wants to choose a consumption program c(t) for 0≤t≤T which will maximize utility

#### U(T) = ∫0T ln(c(t))exp(-at)dt.

The financial assets A(t) of the individual are determined by the differential equation:

#### dA/dt = y(t) + rA(t) - c(t).

There is also a requirement that the financial assets of the individual at the end of his lifespan be nonnegative; i.e.,

#### A(T)≥0.

This can be considered a problem of Bolzano with X1=U and X2=A.

The steering functions are:

#### dU/dt = ln(c(t))exp(-at) dA/dt = rA + y(t) - c(t).

The objective function is to maximize U(T) subject to the constraint that A(T)≥0. The constraint can be satisfied by making the objective function

#### V(T) = U(T) + λA(T)

and choose λ sufficiently large to insure that A(T)≥0.

The Hamiltonian function is

#### H(t) = φUln(c)exp(-at) + φA[rA + y - c],

where the adjoint variables are labled by the name of the corresponding state variable.

The condition for an optimal c(t) is

#### φUexp(-at)/c(t) - φA = 0, or c(t) = φUexp(-at)/φA.

The adjoint variables are defined by

#### dφU/dt = -∂H/∂U = 0 dφA/dt = -∂H/∂A = -φAr.

Since dφU/dt = 0 for all t, φU = constant. Therefore since φU(T)=1, φU(t)=1 for all t.

The equation for φA implies that

so

#### d(ln[φA])/dt = -r

and hence, as a result of integrating from t to T,

Since φ(T) = λ,

#### ln[φA(t)] = ln(λ) + r(T-t) φA = λexp(r(T-t)).

Substituting the values for φU and φA into the condition for an optimal c(t) gives

#### c(t) = exp(-at)/(λ exp(r(T-t)) or, equivalently, c(t) = exp(-(a-r)t)/(λ exp(rT)).

When this expression for c(t) is substituted into the differential equation

#### dA/dt = y(t) + rA(t) - c(t)

and the equation is solved for A(T), a value of λ can be found to make A(T)=0.

## An Example of Finding an Optimal Policy Using Pontryagin's Maximum Principle

Suppose that when there is no fishing the growth of the fish population in a lake is given by

#### dP/dt = 0.08P(1-0.000001P),

where P is the number of fish.

This equation indicates that

dP/dt = 0 when (1-0.000001P)=0; i.e., when P = 1, 000, 000.

Suppose that we want to choose a level of consumption of fish C(t) over the period 0 to T which will maximize the utility

#### U = ∫0T exp(-0.03t)ln(C(t))dt.

This can be put into the form of a problem of Bolzano; i.e., maximize U subject to:

#### dU/dt = exp(-0.03t)ln(C(t)) dP/dt = 0.08P(1-0.000001P)-C(t).

and P(T)≥0.

The constraint P(T)≥0 can be replaced by the requirement that

U(T)+λP(T) be maximized.

The Hamiltonian function is

and hence

#### dφU/dt = 0 and dφP/dt = -φP(0.08)(1 - 0.000002P).

Since φU(T)=1, φU(t) = 1 for all t.

The second equation implies that

#### (1/φP)(dφP)/dt = -(0.08)(1 - 0.000002P) or d(ln(φPP)/dt = -(0.08)(1 - 0.000002P).

The optimal C(t) is the one that maximizes H(t). This is achieved where

#### ∂H/∂C(t) = exp(-0.03t)/C(t) - φP = 0.

Thus the optimal C(t) is given by

#### C(t) = exp(-0.03t)/φP.

The optimal policy is found by solving backwards from t=T the three equations

#### C(t) = exp(-0.03t)/φPdP/dt = 0.08P(1-0.000001P) - C(t) with P(T)=0 d(ln(φP)/dt = -(0.08)/(1 - 0.000002P) with φP(T) = λ.

An approximate solution is determined using

#### dP/dt = [P(t)-P(t-h)]/h so P(t-h) = P(t) - h(dP/dt).

A value of λ is chosen and the value of P(0) is determined. If this value does not equal the given initial value of P then the value of λ is adjusted.