Optimal (but poor if unweighted)
Least-Squares Impulse Response Design

Perhaps the most commonly employed error criterion in signal processing is the least-squares error criterion.

Let $h(n)$ denote some ideal filter impulse response, possibly infinitely long, and let ${\hat h}(n)$ denote the impulse response of a length $L$ causal FIR filter that we wish to design. The sum of squared errors is given by

$$J_2({\hat h}) \isdefs \sum_{n=-\infty}^\infty\left\vert h(n)-{\hat h}(n)\right\vert^2 \eqsp \sum_{n=0}^{L-1}\left\vert h(n)-{\hat h}(n)\right\vert^2 + c_2$$ (5.4)

where $c_2$ does not depend on ${\hat h}$ . Note that $J({\hat h})\geq c_2$ . We can minimize the error by simply matching the first $L$ terms in the desired impulse response. That is, the optimal least-squares FIR filter has the following ``tap'' coefficients:

$${\hat h}(n) \isdef \left\{\begin{array}{ll} h(n), & 0\leq n \leq L-1 \\ [5pt] 0, & \hbox{otherwise} \\ \end{array} \right. \protect$$ (5.5)

The same solution works also for any $\ensuremath{L_p}$ norm (§4.10.1). That is, the error

$$J_p({\hat h}) \isdefs \sum_{n=-\infty}^\infty\left\vert h(n)-{\hat h}(n)\right\vert^p \eqsp \sum_{n=0}^{L-1}\left\vert h(n)-{\hat h}(n)\right\vert^p + c_p \;\geq\; c_p$$ (5.6)

is also minimized by matching the leading $L$ terms of the desired impulse response.

In the $\ensuremath{L_2}$ (least-squares) case, we have, by the Fourier energy theorem (§2.3.8),

$$J_2({\hat h}) \isdefs \sum_{n=-\infty}^\infty\left\vert h(n)-{\hat h}(n)\right\vert^2 \eqsp \frac{1}{2\pi}\int_{-\pi}^{\pi}\left\vert H(\omega)-{\hat H}(\omega)\right\vert^2 d\omega.$$ (5.7)

Therefore, ${\hat H}(\omega)=\hbox{\sc DTFT}({\hat h})$ is an optimal least-squares approximation to $H(\omega)$ when ${\hat h}$ is given by (4.5). In other words, the frequency response of the filter ${\hat H}$ is optimal in the $\ensuremath{L_2}$ (least-squares) sense.