Timeline for How to evaluate/validate clusters using multiple clustering methods
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 13, 2017 at 12:44 | history | edited | CommunityBot |
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
|
|
| Nov 13, 2013 at 20:36 | comment | added | JEquihua | This has a nice answer by the way: stats.stackexchange.com/questions/67686/… | |
| Nov 13, 2013 at 20:22 | comment | added | JEquihua | For example by creating principal components, there are some variations for mixed type data or use multiple correspondence analysis on just your categorical variables and then feeding (all) these to k-means. Also random forest in unsupervised mode (labs.genetics.ucla.edu/horvath/RFclustering/RFclustering.htm) could be quite cool. Problem is i'm not sure your data size will allow these kind of things. | |
| Nov 13, 2013 at 20:21 | comment | added | JEquihua | I'm almost sure clValid has some example for hierarchical clustering. The indices available in fpc for estimating the number of clusters are intended for clustering on continuous type variables. The clusterboot function in the fpc package tries to see how replicable your clustering is on bootstrapped sampes or to data with noise introduced into it, that could be of help to you. It is well documented and points you to this source which is free: Cluster-wise assessment of cluster stability. You could also try methods for mixed-type clustering. | |
| Nov 13, 2013 at 17:30 | comment | added | daniellopez46 | My actual data set only has three continuous variables and about a dozen or so categorical variables. As far as I know the only method I can use is hierarchical. Do you have a good link to something that explains how one can go about evaluating the validity of clusters produced by hierarchical clustering? | |
| Nov 13, 2013 at 15:30 | comment | added | JEquihua | By the way, if you are familiar with k-means, it handles well large-ish data. Maybe you should explore ensembles of just k-means runs. Another thing, mini-batch k-means updates centers based on random samples of data so it is suitable for very large data and is supposed to sometimes converge to the same optimal points (you can check the paper and implementation in the scikit learn python package). There are a few packages in R for cluster validation associated to good theoretical sources: fpc, clValid. You can also maybe look a bit into separability measures like Jeffries-Matusita. | |
| Nov 12, 2013 at 19:22 | vote | accept | daniellopez46 | ||
| Nov 12, 2013 at 19:22 | comment | added | daniellopez46 | thanks for the links to the question you previously answered as well as more info on consensus clustering. I think at this point I will play around with consensus clustering for a small data set I have but will also continue my learning using various individual clustering algorithms. I guess I need to learn more about evaluating clustering algorithm output. | |
| Oct 31, 2013 at 18:03 | history | answered | JEquihua | CC BY-SA 3.0 |