Optimal (but poor if unweighted) Least-Squares
Impulse Response Design

Perhaps the most commonly employed error criterion in signal processing is the least-squares error criterion.

Let $h(n)$ denote some ideal filter impulse response, possibly infinitely long, and let ${\hat h}(n)$ denote the impulse response of a length $L$ causal FIR filter we wish to design. The sum of squared errors is given by

$$J_2({\hat h}) \isdef \sum_{n=-\infty}^\infty\left\vert h(n)-{\ha... ...\right\vert^2 = \sum_{n=0}^{L-1}\left\vert h(n)-{\hat h}(n)\right\vert^2 + c_2$$

where $c_2$ does not depend on ${\hat h}$. Note that $J({\hat h})\geq c_2$. We can minimize the error by simply matching the first $L$ terms in the desired impulse response. That is, the optimal least-squares FIR filter has the following ``tap'' coefficients:

$${\hat h}(n) \isdef \left\{\begin{array}{ll} h(n), & 0\leq n \leq L-1 \\ [5pt] 0, & \hbox{otherwise} \\ \end{array} \right. \protect$$ (E.2)

The same solution works also for any $Lp$ norm. That is, the error

$$J_p({\hat h}) \isdef \sum_{n=-\infty}^\infty\left\vert h(n)-{\ha... ...rt^p = \sum_{n=0}^{L-1}\left\vert h(n)-{\hat h}(n)\right\vert^p + c_p \geq c_p$$

is also miminized by matching the leading $L$ terms of the desired impulse response.

In the $L2$ case, we have, by the Fourier energy theorem (§2.3.8),

$$J_2({\hat h}) \isdef \sum_{n=-\infty}^\infty\left\vert h(n)-{\hat... ...i}\int_{-\pi}^{\pi}\left\vert H(\omega)-{\hat H}(\omega)\right\vert^2 d\omega.$$

Therefore, ${\hat H}(\omega)=\hbox{\sc DTFT}({\hat h})$ is an optimal least-squares approximation to $H(\omega)$ when ${\hat h}$ is given by (E.2). In other words, the frequency response of the filter ${\hat H}$ is optimal in the $L2$ sense.