I have been studying an incremental clustering algorithm for a large set of data that exhibit an inherent dynamic behavior (that is new data can get added over time and some older data may get deleted too based on the current situation). In this scenario, let us assume, some clusters are formed at one particular time instant. Now when new data comes in, the processing should be limited to only handling this new data and and after metadata extraction, they should be clustered into the already existing clusters (if they are similar, otherwise new cluster must be formed). Now, I am really stumped with the problem of analyzing this algorithm. What kind of cluster validity measures can I use to evaluate the goodness of the clusters formed here? I have only started working in the area and so forgive me if this problem has an obvious answer that I have missed.

Thanks in advance. Really hope someone can help me with this.


Cluster validity is a fragile concept. There have been a dozen proposals, but none that is truly convincing.

I recommend to use an application for evaluation.

Only if the clustering helps you solve an actual problem then it was successful. It's meaningless to score high on some artificial metric if you can't apply it to real problems.

So what's your use case scenario? Probably not streaming that dreadful poker hands data set or forest cover, like many of the "stream clustering" people do. Solving problems that do not exist on data that isn't a real stream...

  • $\begingroup$ Thanks for the answer. I am developing a system that performs a focused crawl of the Web to gather news on various topics. A user is able to query the system for news on a particular topic and the response time has to small, so I perform domain specific clustering to reduce the search space. Here, the next crawl can add new documents and we do not want to restart the entire clustering process, but process only changes. My system is currently able to achieve good results. However, I have no idea how to evaluate the goodness of the clusters and hence my clustering algorithm. Hope that helps. $\endgroup$ – Skarth Sep 13 '14 at 8:20
  • $\begingroup$ Can you get user feedback? Usually, with clustering approaches, it's about enabling the user to learn something about the data; so user feedback is the best source of validation data. $\endgroup$ – Has QUIT--Anony-Mousse Sep 13 '14 at 9:54
  • $\begingroup$ Do you mean that I should use external validation measures like F-measure? Allow the user to query the system and calculate precision and recall? $\endgroup$ – Skarth Sep 16 '14 at 11:13
  • $\begingroup$ No, I suggest you produce a complete output, and analyze it at the next stage of your pipeline. Clustering itself is useless. You need to process the clustering output. That is the best way to analyze a clustering result. Don't consider clustering a standalone purpose. So when clustering, say, news articles: present the clusters to the user, and ask the user if they were helpful or not. Don't assume the user could predict the output of the clustering algorithm (= external evaluation). $\endgroup$ – Has QUIT--Anony-Mousse Sep 16 '14 at 13:26
  • $\begingroup$ Now I understand what I should do. Your help is much appreciated. Thank you! $\endgroup$ – Skarth Sep 17 '14 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.