Recently, the diffusion strategy has been used in several applications including
sparse system identification [6] and multitask networks [8] where the adaptive network is assigned to perform estimations for more than one desired variable.
While direct methods can be applied to solve this kind of system at a much lower cost [3], iterative methods are very useful for a large
sparse system. But for small or medium sparse matrix, iterative methods cannot do better than direct methods.
The PNLMS algorithm, which is an NLMS algorithm improved by the use of a proportionate technique, has been proposed for
sparse system identification and echo cancellation.
Mei, "??0 norm constraint LMS algorithm for
sparse system identification," IEEE Signal Processing Letters, vol.
Motivated by the proportionate step-size adaptive filtering [13,14], the proportionate NSAF (PNSAF) has been presented to combat poor convergence in
sparse system identification [15].
Just like the other problems, this is the most time-consuming step, that is, to solve the
sparse system Ax = b.
of nonzero entries, N-size of
sparse system of linear equations, T-a mass matrix, S-a stiffness matrix.
Despite the fact that, in this case, A will in general be dense, the problem can still be considered sparse as all linear algebra operations required involve only sparse matrix multiplication, and
sparse system solves as A never needs to be formed to implement the ADI method; see [5].
First of all the zero-attracting LMS (ZA-LMS) and reweighted ZA-LMS (RZA-LMS) algorithms had been proposed in [27] specially for
sparse system identification and from that time, almost all newly presented sparsity aware algorithms have shared the penalty terms of these algorithms.