|San José State University|
& Tornado Alley
Pontryagin's Maximum Principle applies to a particular type of problem called a Bolzano Problem. Most optimization problems can be put into the form of a Bolzano problem, but more about that later.
A Bolzano problem involves a number of state variables which can change over time where time t runs from 0 to T. Let us suppose the state variables are X1(t),X2(t),...,Xn(t). We want to maximize
given that we start at the point X1(0), X2(0),..., Xn(0), and where the coefficients c1, c2, ...,cn are given and T is some definite finite time. We are given so-called steering functions for controlling the changes in the state variables; i.e.,
dX2/dt = f2(X1,X2,..,
dXn/dt = fn(X1,X2,..,
where the variables u1,u2,...,um are functions of time and are called the control variables. The objective is to choose the control variables at each instant of time so as to steer the state variables from their initial values
to some point
where V(T) = c1X1(T) + c2X2(T) + ...cnXn(T) is maximized.
This seems to be a very difficult task. Pontryagin's Maximum Principle provides a neat, systematic solution.
To implement Pontryagin's method one defines a Hamiltonian function
where the set of adjoint variables φ1,φ2,..,φn are such that
= −Σi φi(∂fi/∂Xj)
and φi(T)= ci for i=1,2,..,n. Note that if H does not depend upon Xj then dφj/dt=0 for all t and thus φjj would be said to be conserved.
The optimum value of the control variables at time t are the ones that maximizes H.
This usually means that the optimum uk(t) is such that
ΣiXi(∂fi/∂uk(t)) = 0
unless uk is constrained, in which case the optimal uk may be at a limit of its range.
Suppose an individual has a noninterest income of y(t) for 0≤t≤T which can either be consumed or saved at an interest rate of r. The individual wants to choose a consumption program c(t) for 0≤t≤T which will maximize utility
The financial assets A(t) of the individual are determined by the differential equation:
There is also a requirement that the financial assets of the individual at the end of his lifespan be nonnegative; i.e.,
This can be considered a problem of Bolzano with X1=U and X2=A.
The steering functions are:
dA/dt = rA + y(t) - c(t).
The objective function is to maximize U(T) subject to the constraint that A(T)≥0. The constraint can be satisfied by making the objective function
and choose λ sufficiently large to insure that A(T)≥0.
The Hamiltonian function is
where the adjoint variables are labled by the name of the corresponding state variable.
The condition for an optimal c(t) is
The adjoint variables are defined by
dφA/dt = -∂H/∂A = -φAr.
Since dφU/dt = 0 for all t, φU = constant. Therefore since φU(T)=1, φU(t)=1 for all t.
The equation for φA implies that
and hence, as a result of integrating from t to T,
Since φ(T) = λ,
φA = λexp(r(T-t)).
Substituting the values for φU and φA into the condition for an optimal c(t) gives
c(t) = exp(-(a-r)t)/(λ exp(rT)).
When this expression for c(t) is substituted into the differential equation
and the equation is solved for A(T), a value of λ can be found to make A(T)=0.
Suppose that when there is no fishing the growth of the fish population in a lake is given by
where P is the number of fish.
This equation indicates that
dP/dt = 0 when (1-0.000001P)=0; i.e., when P = 1,000,000.
Suppose that we want to choose a level of consumption of fish C(t) over the period 0 to T which will maximize the utility
This can be put into the form of a problem of Bolzano; i.e., maximize U subject to:
dP/dt = 0.08P(1-0.000001P)-C(t).
The constraint P(T)≥0 can be replaced by the requirement that
U(T)+λP(T) be maximized.
The Hamiltonian function is
dφP/dt = -φP(0.08)(1 - 0.000002P).
Since φU(T)=1, φU(t) = 1 for all t.
The second equation implies that
or d(ln(φPP)/dt = -(0.08)(1 - 0.000002P).
The optimal C(t) is the one that maximizes H(t). This is achieved where
Thus the optimal C(t) is given by
The optimal policy is found by solving backwards from t=T the three equations
dP/dt = 0.08P(1-0.000001P) - C(t)
d(ln(φP)/dt = -(0.08)/(1 - 0.000002P)
with φP(T) = λ.
An approximate solution is determined using
A value of λ is chosen and the value of P(0) is determined. If this value does not equal the given initial value of P then the value of λ is adjusted.
HOME PAGE OF Thayer Watkins