3
$\begingroup$

Consider the non-negative factorization of a positive, real symmetric matrix A. Non-negative factorization of this matrix yields A = WH where W != H^T.

Yes, there are special cases of A = WW^T factorization, but I understand that this is an imposed constraint. Thus, my question is two-fold:

  1. Intuitively, why does NMF for a symmetric A not yield A = WW^T?
  2. What is the meaning of non-transpose-identity in WH matrices when A is symmetric? How can the orthogonal factors in W and H be interpreted in terms of relationships in A? Is assignment of a loading to W over H a stochastic process? Thus, can conclusions be drawn about the relationships between columns across factors within W or H alone?

My application: I'm trying to decompose gene associations from a gene adjacency matrix. I want to not only decompose the largest sources of variation (i.e. SVD) but also orthogonal and nested gene associations. NMF is ideal for decomposing orthogonal associations as latent factors, and then a similarity generated from relationships across NMF latent factors could enable decomposition of first-order nested gene associations. This sets up an iterative framework for decomposition of higher-order nested gene associations. However, I worry that NMF will randomly partition genetic interaction information due to asymmetry of orthogonal matrices, thus resulting in exponential fragmentation of information with each iteration. I'm at a loss for how to prove or disprove this notion.

$\endgroup$

1 Answer 1

6
$\begingroup$

The reason why $H\neq W^T$ is that for two vector $a$ and $b$ to be orthogonal, you need:

$$\sum_{i=1}^n a_ib_i=0$$

This will typically be achieved by having $a_ib_i$ to be positive at some indices and negative at others - and this is not possible under non-negative matrix factorization. If you factor $A$ as $WW^T$ (maybe with eigendecomposition or Cholesky), the resulting $W$ will almost certainly have negative numbers in it.

As for the interpretation, I'm afraid I can't figure out the correct intuition for NMF of a adjacency matrix. I would advise you not to discard SVD-like approaches, since there are many interesting results in spectral theory applied to adjacency matrices, including spectral graph theory, spectral embeddings and spectral clustering. Those approaches might be well suited to your use case (and info about these methods seems to be way easier to find online).

$\endgroup$
1
  • $\begingroup$ Thanks for that! I've done some systematic testing with real data,it turns out that if A similarity is random, W is not equal to W^T. However, if A similarity richly embeds meaningful latent information, W is strongly correlated with W^T, though latent factors will have different slopes. The extent to which this correlation is linear (in terms of axes units) depends on the algorithm. CoGAPS performed the best in terms of axis units linearity, while nmf default algorithm factors were correlated with the lowest dispersion of slope between factors. Correlation strength was good in all. $\endgroup$
    – zdebruine
    Oct 7, 2020 at 0:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.