Is there an equivalent in NMF to PCA projection? For example lets say you have 2 datasets of data which are generated by a highly similar process, one which is noisy (dataset 2) and one which is not (dataset 1). Lets assume by PCA you can see distinct clusters in dataset 1 by looking at pc1 vs pc2 but not in dataset 2. For PCA you can use the use the loadings from a PCA of dataset#1 with the values from dataset 2 to project it onto dataset1 pca space. This might be in the hopes that it can help you avoid noise in the signal(there are other strategies you could use on dataset 2 alone but i am interested in the projection aspect) . Is there an equivalent strategy for NMF.

Same scenario now with NMF, lets say i did an NMF with equivalent ranks on both datasets(which are now non negative for this example) and now have 2 WxH matrix sets(W1,H1),(W2,H2). Could i equivalently use w1 and the data from something of the form W1xH2 to recreate my dataset then do PCA. I.e use the W1 feature weights along with the sample weights from H2. Is this mathematically acceptable? Essentially I want to use information from dataset 1 to help cluster dataset 2.

  • $\begingroup$ what do you mean by seeing clusters by PCA? PCA is not clustering method. Do you mean seeing clusters if you plot your data based on first 2 principal components? $\endgroup$ – rep_ho Oct 1 '16 at 10:46
  • $\begingroup$ Yes I mean by pc1 vs pc2. I updated question and title. Hopefully it's more clear? $\endgroup$ – SuperCal123 Oct 1 '16 at 17:11

Yes, in both cases, PCA and NNMF will produce components that are linear combinations of other variables. So in the same way as in PCA, you can just project your data based on components from NNMF. Mathematically it is therefore completely acceptable.

That being said, with NNMF you don't have the situation when first few components capture most of the variance of your data. So to project your data based on first 3 components form NNMF doesn't make sense, or it make as much sense as projecting them to any 3 components.

To further illustrate my point, here is a picture from Lee and Seung 1999; Learning the parts of objects by non-negative matrix factorization. enter image description here

If you are trying to classify faces, and use first 7 NNMF components, you may end up classifying them based on a presence of mustache and a nose, neck and ear shadows. But as you commented, you can use all components and get a cleaner version of the data, just because you have fewer components than you had features, or even do some additional feature selection.

| cite | improve this answer | |
  • $\begingroup$ Curious your thoughts, for NMF I thought the whole purpose is to capture the latent "factors" in the data, as well as being a dimensionality reduction technique. I guess I understand your point because unlike PCA, there are no guarantees on the proportion of variance each additional factor captures? In my case I usually try a bunch of ranks (i.e. ks) to get good clustering of dataset 1. Usually even for high dimensional data this k is invariably not large. (and good k is decided usually by cophenetic,silouhette width, or reconstruction error). $\endgroup$ – SuperCal123 Oct 2 '16 at 2:40
  • $\begingroup$ For the NMF case if as you say it is acceptable, I would probably then just use the all factors of the best k that described the first dataset. My reasoning being the W1 matrix is essentially "cleaner" and has a better chance of helping me define clusters that were not well separated using dataset 2 alone . Does that make sense? $\endgroup$ – SuperCal123 Oct 2 '16 at 2:52
  • $\begingroup$ yes, it does. I edited my post $\endgroup$ – rep_ho Oct 2 '16 at 7:41
  • $\begingroup$ Just one question. In the case of NMF, if the obtained components are equal to the number of features, then can I select the first 2-3 and project the day like I would do in the PCA case? If no, why? Aren’t the NMF obtained components ordered ? $\endgroup$ – seralouk Mar 9 '18 at 17:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.