Next  |  Prev  |  Up  |  Top  |  JOS Index  |  JOS Pubs  |  JOS Home  |  Search

### Geometrical Interpretation of Least Squares

Typically, , i.e., the number of frequency constraints is much greater than the number of design variables (filter taps). In these cases, we have an overdetermined system of equations (more equations than unknowns). Therefore, we cannot generally satisfy all the equations, and we are left with minimizing some error criterion to find the optimal compromise'' solution.

In the case of least-squares approximation, we are minimizing the Euclidean distance, which suggests the following geometrical interpretation:

This diagram suggests that the error vector is orthogonal to the column space of the matrix , hence it must be orthogonal to each column in .

This is how the orthogonality principle can be used to derive the fact that the best least squares solution is given by

Note that the pseudo-inverse projects the vector onto the column space of .

In Matlab:

• x = A b
• x = pinv(A) * b

Next  |  Prev  |  Up  |  Top  |  JOS Index  |  JOS Pubs  |  JOS Home  |  Search