Introduction

The optimization (minimization or maximization) of a function of a number of unknown parameters (possibly) subject to constraints is, along with the solution of differential equations and linear systems, one of the three corner-stones in computational applied mathematics. In this course, we aim to introduce the central ideas behind algorithms for the numerical solution of differentiable optimization problems. We intend to present key methods for both unconstrained and constrained optimization, as well as providing theoretical justification as to why they succeed.

The major pre-requisites for the course will be some knowledge of both linear algebra and real analysis, while an appreciation of methods for the numerical solution of linear systems of equations will be helpful.

Intended Synopsis

Brief content of the lectures:

1.	Optimality conditions and why they are important.
2-4.	Line-search methods for unconstrained minimization.
5-7.	Trust-region methods for unconstrained minimization.
8-10.	Interior-point methods for constrained minimization.
11-12.	Sequential quadratic programming (SQP) methods for constrained minimization.

Reading List

The following books contain useful background material:

• J. Dennis and R. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, (republished by) SIAM (Classics in Applied Mathematics 16) 1996

• J. Nocedal and S. Wright, Numerical Optimization, Springer Verlag 1999

• P. Gill, W. Murray and M. Wright, Practical Optimization, Academic Press 1981

• R. Fletcher, Practical Methods of Optimization, 2nd edition Wiley 1987, (republished in paperback 2000)

• A. Conn, N. Gould and Ph. Toint, Trust-Region Methods, SIAM 2000