I'm trying to understand whether this behavior is a bug or a feature.

Essentially, I have a dataset of ten thousand short pieces of text. I have used the CountVectorizer function to turn this into a term-document matrix, and also converted the raw counts to tf-idf scores using TfidfTransformer.

I've then used the NMF function to perform a non negative matrix factorization, as below:

nmf = NMF(n_components=100, random_state=1,
          alpha=.1, l1_ratio=.5)

When I do this with 50 components, the output looks fine. But when I use 100 components, I get a number of components spread throughout the 100 that simply have a 0 weight for every feature (in this case, a word). Now, I know NMF uses a stochastic and iterative optimizer, and I believe it potentially uses some regularisation - so are these zero-weighted topics just indicating that the data suggests less than 100 topics? Or is this a failure of the algorithm, or bug in my code?

  • $\begingroup$ it could be a bug or could be a reasonable answer. Given the doubling of topics you mentioned my question would be if you look at intermediate values between 50 and 100 do you see similar behavior? $\endgroup$ – Lucas Roberts Aug 14 '18 at 22:37

For future readers, it looks like this was due to regularization and not a bug. The number of zero-weighted topics increases as you increase the alpha parameter (which controls the strength of regularization in nmf), and also the number of topics. If you set alpha to zero, you don't get any zero-weighted topics (because no regularisation) at least up to 100 topics in my example.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.