San José State University 

appletmagic.com Thayer Watkins Silicon Valley & Tornado Alley USA 


Pontryagin's Maximum Principle applies to a particular type of problem called a Bolzano Problem. Most optimization problems can be put into the form of a Bolzano problem, but more about that later.
A Bolzano problem involves a number of state variables which can change over time where time t runs from 0 to T. Let us suppose the state variables are X_{1}(t),X_{2}(t),...,X_{n}(t). We want to maximize
given that we start at the point X_{1}(0), X_{2}(0),..., X_{n}(0), and where the coefficients c_{1}, c_{2}, ...,c_{n} are given and T is some definite finite time. We are given socalled steering functions for controlling the changes in the state variables; i.e.,
dX_{2}/dt = f_{2}(X_{1},X_{2},..,
X_{n},u_{1},u_{2},..,u_{m})
...........................................................
dX_{n}/dt = f_{n}(X_{1},X_{2},..,
X_{n},u_{1},u_{2},..,u_{m})
where the variables u_{1},u_{2},...,u_{m} are functions of time and are called the control variables. The objective is to choose the control variables at each instant of time so as to steer the state variables from their initial values
to some point
where V(T) = c_{1}X_{1}(T) + c_{2}X_{2}(T) + ...c_{n}X_{n}(T) is maximized.
This seems to be a very difficult task. Pontryagin's Maximum Principle provides a neat, systematic solution.
To implement Pontryagin's method one defines a Hamiltonian function
where the set of adjoint variables φ_{1},φ_{2},..,φ_{n} are such that
= −Σ_{i} φ_{i}(∂f_{i}/∂X_{j})
and φ_{i}(T)= c_{i} for i=1,2,..,n. Note that if H does not depend upon X_{j} then dφ_{j}/dt=0 for all t and thus φ_{jj} would be said to be conserved.
The optimum value of the control variables at time t are the ones that maximizes H.
This usually means that the optimum u_{k}(t) is such that
Σ_{i}X_{i}(∂f_{i}/∂u_{k}(t)) = 0
for k=1,...,m.
unless u_{k} is constrained, in which case the optimal u_{k} may be at a limit of its range.
Suppose an individual has a noninterest income of y(t) for 0≤t≤T which can either be consumed or saved at an interest rate of r. The individual wants to choose a consumption program c(t) for 0≤t≤T which will maximize utility
The financial assets A(t) of the individual are determined by the differential equation:
There is also a requirement that the financial assets of the individual at the end of his lifespan be nonnegative; i.e.,
This can be considered a problem of Bolzano with X_{1}=U and X_{2}=A.
The steering functions are:
dA/dt = rA + y(t)  c(t).
The objective function is to maximize U(T) subject to the constraint that A(T)≥0. The constraint can be satisfied by making the objective function
and choose λ sufficiently large to insure that A(T)≥0.
The Hamiltonian function is
where the adjoint variables are labled by the name of the corresponding state variable.
The condition for an optimal c(t) is
The adjoint variables are defined by
dφ_{A}/dt = ∂H/∂A = φ_{A}r.
Since dφ_{U}/dt = 0 for all t, φ_{U} = constant. Therefore since φ_{U}(T)=1, φ_{U}(t)=1 for all t.
The equation for φ_{A} implies that
so
and hence, as a result of integrating from t to T,
Since φ(T) = λ,
φ_{A} = λexp(r(Tt)).
Substituting the values for φ_{U} and φ_{A} into the condition for an optimal c(t) gives
or, equivalently,
c(t) = exp((ar)t)/(λ exp(rT)).
When this expression for c(t) is substituted into the differential equation
and the equation is solved for A(T), a value of λ can be found to make A(T)=0.
Suppose that when there is no fishing the growth of the fish population in a lake is given by
where P is the number of fish.
This equation indicates that
dP/dt = 0 when (10.000001P)=0; i.e., when P = 1,000,000.
Suppose that we want to choose a level of consumption of fish C(t) over the period 0 to T which will maximize the utility
This can be put into the form of a problem of Bolzano; i.e., maximize U subject to:
dP/dt = 0.08P(10.000001P)C(t).
and P(T)≥0.
The constraint P(T)≥0 can be replaced by the requirement that
U(T)+λP(T) be maximized.
The Hamiltonian function is
and hence
dφ_{P}/dt = φ_{P}(0.08)(1  0.000002P).
Since φ_{U}(T)=1, φ_{U}(t) = 1 for all t.
The second equation implies that
or d(ln(φ_{P}P)/dt = (0.08)(1  0.000002P).
The optimal C(t) is the one that maximizes H(t). This is achieved where
Thus the optimal C(t) is given by
The optimal policy is found by solving backwards from t=T the three equations
dP/dt = 0.08P(10.000001P)  C(t)
with P(T)=0
d(ln(φ_{P})/dt = (0.08)/(1  0.000002P)
with
φ_{P}(T) = λ.
An approximate solution is determined using
A value of λ is chosen and the value of P(0) is determined. If this value does not equal the given initial value of P then the value of λ is adjusted.
HOME PAGE OF Thayer Watkins 