Concavity is valuable in connection with the Gradient Method of minimizing with respect to .
Definition. The gradient of the error measure is defined as the column vector
Definition. The Gradient Method (Cauchy) is defined as follows.
Given , compute
Some general results regarding the Gradient Method are given below.
Theorem. If is a local minimizer of , and exists, then .
Theorem. The gradient method is a descent method, i.e., .
Definition. , , is said to be in the class if all th order partial derivatives of with respect to the components of are continuous on .
Definition. The Hessian of at is defined as the matrix of second-order partial derivatives,
The Hessian of every element of is a symmetric matrix . This is because continuous second-order partials satisfy
Theorem. If , then any cluster point of the gradient sequence is necessarily a stationary point, i.e., .
Theorem. Let denote the concave hull of . If , and there exist positive constants and such that
By the norm equivalence theorem , Eq. (4) is satisfied whenever is a norm on for each . Since belongs to , it is a symmetric matrix. It is also bounded since it is continuous over a compact set. Thus a sufficient requirement is that be positive definite on . Positive definiteness of can be viewed as ``positive curvature'' of at each point of which corresponds to strict concavity of on .