Next  |  Prev  |  Top  |  Index  |  JOS Index  |  JOS Pubs  |  JOS Home  |  Search

Taylor's Theorem

Theorem. (Taylor) Every functional $ J:\Re ^{\hat N}\mapsto\Re ^1$ in $ {\cal C}_2(\Re ^{\hat N})$ has the representation

$\displaystyle J({\hat \theta}+\eta) = J({\hat \theta}) + J^\prime({\hat \theta}) \eta + \frac{1}{ 2}
\eta^T J^{\prime\prime}({\hat \theta}+\lambda\eta) \eta

for some $ \lambda$ between 0 and $ 1$, where $ J^\prime({\hat \theta})$ is the $ {\hat N}\times 1$ gradient vector evaluated at $ {\hat \theta}\in\Re ^n$, and $ J^{\prime\prime}({\hat \theta})$ is the $ {\hat N}\times{\hat N}$ Hessian matrix of $ J$ at $ {\hat \theta}$, i.e.,
$\displaystyle J^\prime({\hat \theta})$ $\displaystyle \mathrel{\stackrel{\mathrm{\Delta}}{=}}$ $\displaystyle \frac{\partial J(\theta)}{\partial \theta}({\hat \theta})$ (5)
$\displaystyle J^{\prime\prime}({\hat \theta})$ $\displaystyle \mathrel{\stackrel{\mathrm{\Delta}}{=}}$ $\displaystyle \frac{\partial^2 J(\theta)}{\partial {\hat \theta}^2}({\hat \theta})$ (6)

Proof. See Goldstein [2] p. 119. The Taylor infinite series is treated in Williamson and Crowell [7]. The present form is typically more useful for computing bounds on the error incurred by neglecting higher order terms in the Taylor expansion.

Next  |  Prev  |  Top  |  Index  |  JOS Index  |  JOS Pubs  |  JOS Home  |  Search

Download gradient.pdf
Download the whole thesis containing this material.

``Elementary Gradient-Based Parameter Estimation'', by Julius O. Smith III, from ``Techniques for Digital Filter Design and System Identification, with Application to the Violin,'' Julius O. Smith III, Ph.D. Dissertation, CCRMA, Department of Electrical Engineering, Stanford University, June 1983.
Copyright © 2006-01-03 by Julius O. Smith III
Center for Computer Research in Music and Acoustics (CCRMA),   Stanford University
CCRMA  [Automatic-links disclaimer]