Mahout Scability Do you know any real world examples of how much Mahout can scale? I wonder how much it can scale in collaborative filtering, clustering, and classification ?
 A: It's not only about the scalability of Mahout. But the scalability of the model used by the algorithm is just as much a question.
K-means is good for teaching clustering, and it works surprisingly often despite being based on some very limiting assumptions - such as Voronoi cell shaped clusters.
But when you scale this up to big data, these things totally get out of control. With k-means, it is already difficult to choose $k$. But if you scale it up to a data set that is 100 times as big, you likely also should increase $k$. To 10x-100x the $k$. But this will make choosing the right $k$, a good initialization, doing enough itertions to get a sound result etc. a lot harder.
So while Mahout enables you to run k-means on much larger data sets, k-means itself may not be appropriate for data sets of this size anymore. Because the logic of k-means is just so much more likely to be flawed on this amount of data.
Btw, I've seen reports here on stackexchange/stackoverflow that k-means in Mahout might be buggy when run on more than one node. Some user reported that even with the same initialization, he keeps on getting different results when running one a different number of nodes. As the implementation supposedly is exact and only depends on the initialization, this should not happen.
