Not very known but effective data mining algorithms? I recently came a cross Matrix Factorization for classification or recommendations. I was very shocked that it was one of the most effective techniques in the Netflix competition. But actually I think very few people have heard about it. Other techniques like SVM, ANN ... etc are very popular. I was wondering if there are other machine learning algorithms that are not popular or very known but showed far better performance than other popular techniques in some domains or problems.
 A: An algorithm that I don't often see in most data mining literature, that works very well on large data mining numerical prediction problems is cubist
There is also an R based package.  From the R vignette:
"Cubist is a rule based model that is an extension of Quinlan's M5 model tree. A tree is grown where
the terminal leaves contain linear regression models. These models are based on the predictors used
in previous splits. Also, there are intermediate linear models at each step of the tree. A prediction
is made using the linear regression model at the terminal node of the tree, but is smoothed by
taking into account the prediction from the linear model in the previous node of the tree (which
also occurs recursively up the tree). The tree is reduced to a set of rules, which initially are paths
from the top of the tree to the bottom. Rules are eliminated via pruning and/or combined for
simplication."
One text that does cover it well is Applied Predictive Modeling, by Max Kuhn and Kjell Johnson. They cover and compare many different predictive models on several problems, and cubist was also often shown as one of the top performers.
A: Matrix factorization is not so much an "algorithm", but a task, and it is IMHO very common.
I've seen whole tracks on matrix factorization at major conferences. For example:


*

*http://kdd2012.sigkdd.org/tutorials.shtml

*http://kdd2012.sigkdd.org/program_research_track.shtml#r7
As you have noticed, it performed very well in some competitions, and since then has become a widely employed technique, with tons of research directed that way. MAybe you just did not recognize its omnipresence, because it is often shortened to "NMF" for example. People assume that everybody recognized NMF as "nonnegative matrix factorization".
I cannot agree that "very few people have heard of it", on the contrary.
I consider it a bit boring, though. Essentially, it's PCA/SVD revisited for recommender systems.
