Effect of Measurement Noise

In practice, measurements are never perfect. Let $\hat{\underline{y}}=\underline{y}+\underline{e}$ denote the measured output signal, where $\underline{e}$ is a vector of ``measurement noise'' samples. Then we have

$$\hat{\underline{y}}= \underline{y}+ \underline{e}= \mathbf{x}\underline{h}+ \underline{e}.$$

By the orthogonality principle [38], the least-squares estimate of $\underline{h}$ is obtained by orthogonally projecting $\hat{\underline{y}}$ onto the space spanned by the columns of $\mathbf{x}$ . Geometrically speaking, choosing $\underline{h}$ to minimize the Euclidean distance between $\hat{\underline{y}}$ and $\mathbf{x}\underline{h}$ is the same thing as choosing it to minimize the sum of squared estimated measurement errors $\vert\vert\,\underline{e}\,\vert\vert ^2$ . The distance from $\mathbf{x}\underline{h}$ to $\hat{\underline{y}}$ is minimized when the projection error $\underline{e}=\hat{\underline{y}}-\mathbf{x}\underline{h}$ is orthogonal to every column of $\mathbf{x}$ , which is true if and only if $\mathbf{x}^T\underline{e}=0$ [84]. Thus, we have, applying the orthogonality principle,

$$0 = \mathbf{x}^T\underline{e}= \mathbf{x}^T(\underline{y}- \mathbf{x}\underline{h}) = \mathbf{x}^T\underline{y}- \mathbf{x}^T\mathbf{x}\underline{h}.$$

Solving for $\underline{h}$ yields Eq.$\,$ (F.8) as before, but this time we have derived it as the least squares estimate of $\underline{h}$ in the presence of output measurement error.

It is also straightforward to introduce a weighting function in the least-squares estimate for $\underline{h}$ by replacing $\mathbf{x}^T$ in the derivations above by $\mathbf{x}^TR$ , where $R$ is any positive definite matrix (often taken to be diagonal and positive). In the present time-domain formulation, it is difficult to choose a weighting function that corresponds well to audio perception. Therefore, in audio applications, frequency-domain formulations are generally more powerful for linear-time-invariant system identification. A practical example is the frequency-domain equation-error method described in §I.4.4 [78].