0
$\begingroup$

I apply L1-norm as a group sparsity constraint [1,2] into non-negative matrix factorization $V \approx WH$ for source separation.

Objective functions:

Standard NMF (Kullback-Leibler divergence): $D_{kl}(W,H,\pi)$

$= \sum_{ij} (-V_{ij} \sum_{k} \pi_{ijk} log \frac{W_{ik}H_{kj}}{\pi_{ijk}} + \sum_{k}W_{ik}H_{kj})$

L1-regularized NMF: $D_{kl}(W,H,\pi) + \lambda\Omega(H) $

$= \sum_{ij} (-V_{ij} \sum_{k} \pi_{ijk} log \frac{W_{ik}H_{kj}}{\pi_{ijk}} + \sum_{k}W_{ik}H_{kj}) + \lambda \sum_{kj}(\frac{H^2_{kj}}{2 \gamma_{kj}} + \frac{\gamma_{kj}}{2})$

Here, my $W$ is fixed with group structure of source (${W}=[{W}^{(1)}, \cdots, {W}^{(G)}]$ ) and only $H$ is updated.

I calculate the result by comparing sum of activation from $H$ matrix of each source group. When the orange is true major source, I found the standard NMF (bottom) can get the higher activation for orange source than L1-regularized NMF (top). Why is this happened? What can be the reasons that L1 is not helpful in this source separation? I think L1 is always useful for sparse model.

enter image description here

$\endgroup$
  • $\begingroup$ Can you be more explicit in your question? Can you add some equations, explain what your data is and what you want to accomplish? Also what do you mean by "the optimization is worse"? Even the term L1 regularization is ambiguous in this context... $\endgroup$ – air Feb 12 '18 at 7:32
  • $\begingroup$ @air thanks for comment. I add the info as you suggested. It's quite a new thing for me, so sorry for lack of good explanation. $\endgroup$ – Jan Feb 13 '18 at 7:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.