9
$\begingroup$

I am wondering why matrix factorization techniques in the machine learning domain almost always expect the provided matrix to be non-negative. What is the advantage of this constraint?

Background: I want to use matrix factorization algorithms for a sparse user-item matrix containing positive and negative implicit feedback. Is there any another possibility to set interactions with negative indications apart from fields that denote that no interaction happened between the user and the item?

$\endgroup$
3
  • 3
    $\begingroup$ In a word: Interpretability. In many cases when trying to model some particular phenomenon negative values are nonsensical or simply impossible. Negative height for people, below zero temperature in Kelvin degrees, negative number of visits in a website, etc. are just a few otherwise perfectly reasonable continuous or discrete variables that are simply nonsensical in terms of a final estimate. $\endgroup$
    – usεr11852
    Feb 20 '15 at 1:21
  • 1
    $\begingroup$ Expanding upon the comment above, I'll mention an application I know relating to gene expression. The factors in this context are interpreted to be sets of genes that express themselves together, and the numbers represent the relative strength of the expression. There is no such thing as negative expression of a gene. $\endgroup$
    – Paul
    May 11 '17 at 17:52
  • 4
    $\begingroup$ In the paper that first made NMF take off, the notion was that nonnegative constraints unexpectedly yielded a "parts-based" decomposition of data, such as face images, into natural, sparse components. lsa.colorado.edu/LexicalSemantics/seung-nonneg-matrix.pdf $\endgroup$
    – Paul
    May 11 '17 at 17:56
2
$\begingroup$

Take a look at Jester, it is a well known data set of jokes which uses continuous ratings in the range of -10.0 to +10.0. A lot of papers have used this dataset for matrix factorization techniques with no ill effects. There is no such positive-ratings-only constraint on a mathematical or technical level.

But the reason we typically see majorly positive rating matrices, has likely something to do with websites/services not wanting their products to have negative ratings: even a one-star somehow appears better than a negative rating, it's how liking something less compares to hating something.

It may also have to do with how users perceive averages: if you use negative and positive ratings, some products could on average have 0 stars, which actually means neutral on that rating scale. But if you're used to the 5-star rating scales used on other websites, neutral is 3 stars and 0 stars would mean "really bad".

$\endgroup$
0
$\begingroup$

Some good points have been raised above. I'll add three ways in which non-negative constraints improve capture of interpretable factor models:

Non-negativity encourages imputation of missing signal. Because signals can only be explained in one direction--additively--much more missing signal is imputed than in SVD, for example. If negative values are permitted in the models, the algorithm may be "tempted" to subtract away real data points rather than adding to missing points. It is often the case that data in the matrix to be factorized is not complete due to signal dropout or incomplete sampling, thus imputation is important.

Non-negativity enforces sparsity. While unconstrained decompositions are almost never zero, non-negative models are always zero where a given factor absolutely does not contribute to a signal. Sparse representations are desirable when we want to discover distinct feature sets or sample relationships.

Non-negativity ensures that factors will not cancel one another out. For instance, if one factor "overcorrects" a signal, another factor my try to "counter-correct" to balance it out. When factors are positive or zero only, they can never counter-correct, and only can explain additive signal. Generally, in a well-conditioned non-negative input matrix, an unconstrained orthogonal factorization will nearly be non-negative. However, imposing non-negativity enforces reasonable theoretical expectations and can accelerate convergence.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.