Typically, , i.e., the number of frequency constraints is much greater than the number of design variables (filter taps). In these cases, we have an overdetermined system of equations (more equations than unknowns). Therefore, we cannot generally satisfy all the equations, and we are left with minimizing some error criterion to find the ``optimal compromise'' solution.
In the case of least-squares approximation, we are minimizing the
Euclidean distance, which suggests the following geometrical
interpretation:
This diagram suggests that the error vector
is orthogonal to
the column space of the matrix
, hence it must be orthogonal to each
column in
.
This is how the orthogonality principle can be used to derive the fact that the best least squares solution is given by
Note that the pseudo-inverse projects the vector onto the column space of .
In Matlab: