Elementary Gradient-Based Parameter Estimation

JULIUS O. SMITH III

Center for Computer Research in Music and Acoustics (CCRMA)
Department of Music, Stanford University, Stanford, California 94305 USA

Abstract:

This section defines some of the basic terms involved in optimization techniques known as gradient descent and Newton's method. Terms defined include metric space, linear space, norm, pseudo-norm, normed linear space, Banach space, Lp space, Hilbert space, functional, convex norm, concave norm, local minimizer, global minimizer, and Taylor series expansion.

Detailed Contents (and Navigation)

• Vector Space Concepts
  • Specific Norms
  
  
• Concavity (Convexity)
  • Concave Norms
  
  
• Gradient Descent
• Taylor's Theorem
• Newton's Method
• Index
• Bibliography
• About this document ...