Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm using Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient. The result (I use 2 as rank) is like this:

Original (Matrix V)
[[ 10  20   1   9]
 [  1   2   1  10]
 [  7   6   1   5]
 [  5   4   1   5]
 [100 200   1  90]]

Basis matrix (Matrix W)
[[ 0.0211552   0.09923254]
 [ 0.032219    0.00224676]
 [ 0.0259072   0.02522768]
 [ 0.02491072  0.01388348]
 [ 0.19794036  1.00108155]]

Coef (Matrix H)
[[ 144.00101426   66.1643417    62.06074785  170.96878852]
 [  69.14663274  187.0342459     0.           54.62548645]]

The Coef seems comparable. For example, in the first column of Coef, the 144 is greater than 69. Does that mean in the first column of original data, the Feature1 is more important than Feature2?

I'm not sure about the previous question. Because the result might also be like this:

Basis matrix (Matrix W)
[[ 0.211552   0.09923254]
 [ 0.32219    0.00224676]
 [ 0.259072   0.02522768]
 [ 0.2491072  0.01388348]
 [ 1.9794036  1.00108155]]
Coef (Matrix H)
[[ 14.400101426   6.61643417    6.206074785  17.096878852]
 [  69.14663274  187.0342459     0.           54.62548645]]

The error function (something like V-WH) might be the same. However, in this situation, in the first column of Coef, Feature2 becomes more important.

I think there might be some constraints for Matrix W and Matrix H to make one of them in the same order of magnitudes so that the Coef is comparable. I checked the paper about Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient, but I can't say I understand the algorithm.

Does anyone have ideas about this?

share|improve this question

You are right on that without proper normalization the columns of W and the rows of H are not comparable. In fact, NMF is invariant with column scaling on W or row scaling on H. Consider a diagonal matrix $S$ with size $k \times k$, $WH$ actually is equal to $WSS^{-1}H$.

Therefore, to compare entries in the $H$, the best practice is to normalize $W$ on columns, e.g. $l_1$-normalization, $\sum_jW_{ij}=1$ or $l_2$-normalization, $\sum_jW_{ij}^2=1$. After normalizing $W$ with $S_{ii} = \sum_jW_{ij}$, make sure to scale $H$ using $S_{ii}^{-1}$ as well.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.