Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I want to cluster ~22000 points. Many clustering algorithms work better with higher quality initial guesses. What tools exist that can give me a good idea of the rough shape of the data?

I do want to be able to choose my own distance metric, so a program I can feed a list of pairwise distances to would be just fine. I would like to be able to do something like highlight a region or cluster on the display and get a list of which data points are in that area.

Free software preferred, but I do already have SAS and MATLAB.

share|improve this question
add comment

8 Answers

up vote 10 down vote accepted

GGobi (http://www.ggobi.org/), along with the R package rggobi, is perfectly suited to this task.

See the related presentation for examples: http://www.ggobi.org/book/2007-infovis/05-clustering.pdf

share|improve this answer
    
Thanks for the suggestion, @Shane. ggobi looks promising, I am installing it right now and will give it a try :) –  anonymous Aug 10 '10 at 15:12
1  
Works fine on other platforms, but gtk does not play nice with OSX. –  anonymous Aug 10 '10 at 17:02
3  
gtk is fine on OSX. –  hadley Jan 21 '11 at 5:44
add comment

Exploring clustering results in high dimensions can be done in R using the packages clusterfly and gcExplorer. Look for more here.

share|improve this answer
    
Thanks, but is there any benefit to using clusterfly rather than calling ggobi directly? The website only mentions clustering methods, which are interesting, but not my primary goal just yet. gcexplorer has less informative website, but looks like it is for visualizing data after it has already been split to clusters. I will give them a try once I get to that point, but not what I need right now. –  anonymous Aug 10 '10 at 15:12
add comment

(Months later,) a nice way to picture k-clusters and to see the effect of various k is to build a Minimum Spanning Tree and look at the longest edges. For example,

alt text

Here there are 10 clusters, with 9 longest edges 855 899 942 954 1003 1005 1069 1134 1267.
For 9 clusters, collapse the cyan 855 edge; for 8, the purple 899; and so on.

The single-link k-clustering algorithm ... is precisely Kruskal's algorithm ... equivalent to finding an MST and deleting the k-1 most expensive edges.

— Wayne, Greedy Algorithms.

22000 points, 242M pairwise distances, take ~ 1 gigabyte (float32): might fit.

To view a high-dimensional tree or graph in 2d, see Multidimensional Scaling (also from Kruskal), and the huge literature on dimension reduction. However, in dim > 20 say, most distances will be near the median, so I believe dimension reduction cannot work there.

share|improve this answer
add comment

I've had good experience with KNIME during one of my project. It 's an excellent solution for quick exploratory mining and graphing. On top of that it provides R and Weka modules seamless integration.

share|improve this answer
    
Looks like a useful program, but their webpage does not do a good job of convincing me it will solve this exact problem. It looks like it may be too broad, too many features I don't care about, making it hard to do the simple things. I will give it another look if the other choices don't work out. –  anonymous Aug 10 '10 at 15:19
add comment

Also have a look at ELKI, an open-source data mining software. Wikimedia commons has a gallery with images produced with ELKI, many of which are related to cluster analysis.

share|improve this answer
    
(+1) Looks great. –  chl Dec 6 '11 at 9:35
add comment

Take a look at Cluster 3.0. I'm not sure if it will do all you want, but it's pretty well documented and lets you choose from a few distance metrics. The visualization piece is through a separate program called Java TreeView (screenshot).

share|improve this answer
    
Thanks for the suggestion, but the ability to choose my own measure of distance is critical, so this won't work to me. Someone else may find it useful, though. –  anonymous Aug 10 '10 at 15:06
add comment

GGobi does look interesting for this. Another approach could be to treat your similarity/inverse distance matrices as network adjacency matrices and feeding that into a network analysis routine (e.g., either igraph in R or perhaps Pajek). With this approach I would experiment with cutting the cutting the node distances into a binary tie at various cutpoints.

share|improve this answer
    
I thought of this but there doesn't seem to be a reasonable cut point, and domain experts can't justify one either. –  anonymous Aug 10 '10 at 15:15
    
I would think this could be fairly arbitrary for your stated purpose - honestly, you might not even need to actually cut into binary, just recode a tie value label on a scale of 1 to some manageable number, then progressively hide/show the ties at various levels (optionally also hiding/eliminating any pendants & orphans along the way). Not directly responding to your request as written, but why not take a more typical approach and use a hybrid clustering method that doesn't use initial centroids to identify preliminary clusters, then feed the centroids from that result into your new analysis? –  Shelby Aug 10 '10 at 16:45
    
I am guessing you mean to try for many different cutoffs until I see some nice results? I wish to avoid that for standard multiple comparisons reasons. re: your second suggestion I guess I just trust myself better than those algorithms. I use the computer to process large amounts of data too tedious to do by hand, not to replace my thinking. –  anonymous Aug 10 '10 at 17:06
1  
You're using hypothesis testing language but yet talking about a very exploratory, know-it-when-you-see-it approach @ the same time - so it's not clear what your goal really is for this part of your analysis. If you have hypotheses you're testing later (e.g. predicting cluster membership or using clust membership as predictor) then you can choose not to do things that will tempt bias there. But "multiple comparison" issues don't really figure into the exploratory problem you're describing. The viz cutoffs are just to help you see what's there - but your trust may still be misplaced. –  Shelby Aug 11 '10 at 11:48
add comment

Weka is an open source program for data mining (wirtten and extensible in Java), Orange is an open source program and library for data mining and machine learning (written in Python). They both allow convenient and efficient visual exploration of multidimensional data

share|improve this answer
    
Orange's features page says 'under construction' and they don't list screenshots like what I am doing. weka has no features list at all. They may be able to do what I want, but if they don't promote the feature, how can I tell. I am more convinced by the other choices. –  anonymous Aug 10 '10 at 15:24
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.