Summary on project literature review

Blind Source Separation for Convolutive Mixtures

The existing approaches to BSS for convolutive mixtures can be categorised broadly into several areas, such as, time-domain and frequency domain approaches, batch and sequential approaches. This summary is an attempt to inspect, distinguish and categorise the approaches with respect to the principles upon which some kind of optimisation will be applied. Other distinguishable aspects will be stated if known in the context.

Information-theoretic approach

In [1], Bell and Sejnowski proposed an information-theoretic approach to solve for the optimal solution of separating matrix. They maximise the mutual-information that an output y=g(u) of a neural processor contains about its input u, where g is a non-linear function and u = Wx. The way to maximise this is by following the natural gradient, in contrast to the absolute gradient as in the classical optimisation in Euclidean space (See comparison of natural gradient, stochastic gradient and quasi-Newton method in [10]). Torkkola then used this approach to design a feedforward and feedback network for mixing inversion in the case of delayed sources and convolved sources [2][3]. However, the feedforward architecture is not sufficient to do the separation. Moreover, it also whitens the signals, which is sometimes undesirable, for example, in the case of speech. The more elegant feedback architecture performs better and the whitening problem can be avoided by forcing the direct filters to mere scaling coefficients, with indeterminacy of the recovered sources up to permutation and filtering. However, it still cannot deal with a non-minimum-phase mixing matrix. The non-minimum-phase situation may arise when the echo picked up by sensors or microphones is stronger than the direct signal. This will cause the solution matrix filters to be non-causal. Lee et al applied the FIR polynomial matrix techniques in the frequency domain described by Lambert[4]. They then arrived at a feedforward architecture which is capable of reversing the non-minimum-phase mixing system and hence can work more generally in real situation [5].

Another algorithm based on maximum entropy formulation is proposed in [9] where the cost function used is based on the entropy of the output. It uses natural gradient optimisation and is proved to be equivarient. The source signals are estimated directly using a truncated version of a doubly-infinite multichannel equalizer/separator of the form

where y(k) is a vector of outputs at time k and x is a vector of signals presented at the sensors.

Density matching approach

A combination of density shape matching approach, using sigmoid function[6], and a context-sensitive generalisation of ICA[7] was applied to the feedforward separating system[8]. The latter is an extension of the infomax ICA algorithm where the underlying probability density is estimated as a weighted sum of logistic functions with some dependency on the history of the source.

A similarly recurrent structure but with different approach in solving for optimal solution was proposed by Nguyen and Jutten [17]. It is based on a cross-filter feedback network whose weights are updated using two different non-linear functions operating on the two estimates of the two sources (2-D case). The algorithm was then taken up by Charkani and Deville [18] who studied the stability and arrived at an asymptotic error variance of the separating system minimisation approach. This, however, requires the knowledge of probability density in order to find a suitable separating function used in the algorithm. Therefore, self- adaptive optimisation of separating functions is adopted by estimating the best projection of the optimum separating functions on a given set of functions. There is no restriction on sources.

Dynamic Component Analysis [16] is another algorithm proposed for a separation of the convolutive mixtures. It is based on a minimisation of Kullback-Liebler distance which is widely used to measure the deviation of two probability density distributions. Here, the distance is considered block-wise between the model sensor distribution, parameterised by the unmixing filter parameters, and the observed distribution. The distance is then minimised subject to those parameters using the stochastic gradient method which can be done in either time and frequency domain. It exploits high order temporal and inter-sensor statistics to achieve a separation.

Exploitation of non-stationarity, using 2^nd-order statistics

An alternative approach to achieve statistical independence condition, using additional 2^nd order information of nonstationary signals, has been proposed by Parra and Spence[11]. While it is true that 2^nd order statistics at single time is not sufficient, additional information can be obtained if one considers 2^nd order statistics at multiple times. This has been used in an instantaneous case [12], by decorrelating(diagonalising) the covariance matrix at several times, and the equivalent problem of convolutive mixtures of narrow-band signals [13]. They then took up the multiple decorrelation approach assuming nonstationary signals and use a least squares optimisation to estimate A or W. Also, the noise power could be estimated provided the number of sensors are greater than or equal to the number of sources. The cost function is derived from a careful consideration of measurement of 2^nd order statistics in the frequency domain. The algorithm also works in frequency domain and the problem of permutation of frequency bins at several times is avoided by a constraint on the filter length. A similar approach was undertaken by Principe in frequency-domain[14] and Kawamoto in time-domain[15].

Simultaneous Diagonalisation(SD), either in time or frequency domain, uses the same approach as above which can be used to solve the BSS problem. Frobenius norm is utilised as a measurement of the simultaneous diagonalisation for BSS in time-domain [21] and has been extended to frequency domain in [22]. The approach in[20], an algorithm based on frequency decomposition and SD is derived. It is also based on Hadamard’s inequality in order to make the algorithm more computationally efficient than existing SD algorithms.

In frequency domain, there are problems which need to be taken care of. They are scaling and permutation and the computational complexity. However, it has the advantage over time-domain solution when the separating filters are long in length.

Deflation Approach/Iterative approach

The deflation approach consists in extracting the sources one by one. More precisely, G(z) is calculated such that, r(n)=[G(z)]y(n) coincides with a filtered version of source, say, s₁(n). Then T(z)is found such that v(n)=y(n)-[T(z)]r(n) has minimum variance. Since r(n) is supposed to be independent of s_i(n), i¹1, therefore, v(n) is a convolutive mixture of s_i(n), i¹1 and a filtered version of s₁ has been extracted as r(n). Other sources can be extracted using the same iterative scheme. In [23], it is shown that a maximisation of standard contrast function, such as the normalised fourth-order cumulant, using the deflation approach can yield a separation.

Constrained CMA in hierarchical architecture

In [24], CM algorithm(CMA) is considered in the case of separation of convolutive mixtures. It is used in a combination with hierarchical architecture used in globally convergent approach and a self-organising filter structure using dynamical stability properties exhibited by lateral inhibition mechanisms. The globally convergent approach based on hierarchical architecture forces each filter to restore a source that the other filters are not selecting. In self-organising approach, it is used with measures of the success of a filter to restore a source(and not by others) such that inhibition in the decision rule is increased when outputs of filters are correlated and unsuccessful. This is applied as a constraint prior to using CMA as normal which results in several filters restoring different sources. Both the cross-filter weights and measure of success are adapted recurrently. The algorithm is robust to abrupt changes in the mixture composition.

Adaptive algorithm based on higher-order statistics

A time-domain adaptive algorithm which can separate convolutive mixtures in an independent zero-mean Gaussian noise is proposed in [25]. It is based on a maximisation of normalised fourth-order cumulant with respect to C(z) when

u(k) = C(z)x(k)

where x(k) is the current data vector and C(z) is the equaliser matrix

However, it assumes that the sources are stationary, the number of observations is greater than or equal to the number of sources and uses doubly-infinite equalisers.

Measure of performance(guideline for general models)

See [19].

Reference:

[1] A.Bell and T.Sejnowski, “Blind separation and blind deconvolution: an information-theoretic approach,” in Acoustics, Speech and Signal Processing, 1995. ICASSP-95, Inter. Conf. Vol.5, pp.3415-3418, 1995. (See [2]).

[2] K. Torkkola, “Blind separation of delayed sources based on information maximization,” in Proc. ICASSP'96, pages 3509-3512, Atlanta, Georgia, 1996.

[3] K. Torkkola, “Blind separation of convolved sources based on information maximization,” in Neural Networks for Signal Processing [1996] VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop , 1996 , Page(s): 423 –432.

[4] R. H. Lambert, “Multichannel Blind Deconvolution: FIR Matrix Algebra and Separation of Multipath Mixtures,” PhD thesis, Univ. of Southern California, 1996.

[5] T.Lee, A.Bell and R.Lambert, “Blind separation of delayed and convolved sources,” in Proc. NIPS 96, 1997.

[6] Z.Roth and Y.Baram. “Multidimensional density shapping by sigmoids”, IEEE Trans. On Neural Networks, 7(5):1291-1298, 1996. (See [8])

[7] B.Pearlmutter and L.Parra. A context-sensitive generalization of ICA. In ICONIP’96. In Press.(See [8]).

[8] Te-Won Lee; Orglmeister, R., “A contextual blind separation of delayed and convolved sources,” Acoustics, Speech, and Signal Processing, 1997. ICASSP-97., 1997 IEEE International Conference on Volume: 2 , 1997 , Page(s): 1199 -1202 vol.2

[9] Douglas, S.C.; Cichocki, A.; Amari, S. and A.A.Yang, “Multichannel blind deconvolution using the natural gradient,” in Proc. 1^st IEEE Workshop Signal Processing Applications Wireless Commmunications, 1997, pp. 101-104.

[10] L.Parra, “Temporal models in blind source separation,” in Adaptive Processing of Sequences and Data Structures, L.Giles and M.Gori Eds, Berlin Germany:Springer, 1998, pp. 229-247.

[11] Parra, L.; Spence, C. “Convolutive blind separation of non-stationary sources,” Speech and Audio Processing, IEEE Transactions on Volume: 8 3 , May 2000 , Page(s): 320 –327.

[12] See [11] (25)

[13] See [11] (26)

[14] See [11] (30)

[15] See [11] (31)

[16] N. Attias, C.E. Schreiner, “Blind source separation and deconvolution by dynamic component analysis,” Neural Networks for Signal Processing [1997] VII. Proc. IEEE Workshop, 1997, pp.456-465.

[17] H.L. Nguyen Thi and C. Jutten, “Blind source separation for convolutive mixtures,” Signal Proc., vol.45, no.2, pp. 209-229, March 1995.

[18] N.Charkani, Y. Deville, “A convolutive source separation method with self-optimising non-linearities”, Acoustics, Speech and Signal Processing, 1999. Proc. 1999 IEEE International Conf. On, vol.5, 1999, pp. 2909-2912.

[19] D.W.E. Schobben, K. Torkkola and P. Smaragdis, “Evaluation of Blind Signal Separation Methods,” Proceedings Int. Workshop Independent Component Analysis and Blind Signal Separation, Aussois, France, January 11-15, 1999, pp. 261-266.

[20] J.Principe, H.C. Wu, “Blind separation of convolutive mixtures,” Neural Networks 1999. IJCNN’99 vol.2, 1999, pp. 1054-1058.

[21] See [20]

[22] See [20]

[23] C.Simon, Ph.Loubaton, C.Vignat, C.Jutten and G. d’Urso, “Separation of a class of convolutive mixturesa contrat: a contrast function approach,”

[24] Fijalkow-I; Gaussier-P, “Self-organizing blind MIMO deconvolution,”, Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics. SPW-HOS '99. IEEE Comput. Soc, Los Alamitos, CA, USA; 1999; xii+406 pp. p.300-4, 1999.

[25] J.K. Tugnait, “Adaptive blind separation of convolutive mixtures of independent linear signals,” in Acoustics, Speech and Signal Processing, 1998. Proc. 1998 IEEE Inter. Conf. Vol.4, pp.2097-2100, 1998.