An optimization problem is more complicated if it is constrained, i.e., the arguments of the objective function are subject to certain equality or inequality constraints in terms of what values they can take. Such a constrained optimization problem can be formulated as:

(172) |

In the most general case where the objective function
and the constraint functions
and
are
nonlinear, the process of solving this nonlinear optimization problem
is called *nonlinear programming (NLP)*. Specially, if the
objective function is quadratic while the constraints are linear,
(the feasible region is a polytope), the process is called
*quadratic programming (QP)*. More specially, if the objective
function is linear as well as the constraints, the process is called
*linear programming (LP)*. In the following, we will first
consider the NLP in general, and then discuss more specifically
QP and LP.

- Optimization with Equality Constraints
- Optimization with Inequality Constraints
- Duality and KKT Conditions
- Linear Programming (LP)
- The Simplex Algorithm
- Quadratic Programming (QP)
- Interior Point Methods