Visualization software for clustering I want to cluster ~22000 points. Many clustering algorithms work better with higher quality initial guesses. What tools exist that can give me a good idea of the rough shape of the data?
I do want to be able to choose my own distance metric, so a program I can feed a list of pairwise distances to would be just fine. I would like to be able to do something like highlight a region or cluster on the display and get a list of which data points are in that area.
Free software preferred, but I do already have SAS and MATLAB. 
 A: Exploring clustering results in high dimensions can be done in R using the packages clusterfly and gcExplorer. Look for more here.
A: (Months later,)
a nice way to picture k-clusters and to see the effect of various k
is to build a
Minimum Spanning Tree
and look at the longest edges. For example,

Here there are 10 clusters, with 9 longest edges
855 899 942 954 1003 1005 1069 1134 1267.
For 9 clusters, collapse the cyan 855 edge; for 8, the purple 899;
and so on.

The single-link k-clustering algorithm ... is precisely Kruskal's algorithm
  ... equivalent to finding an MST and deleting the k-1 most expensive edges.

— Wayne, 
Greedy Algorithms.
22000 points, 242M pairwise distances, take ~ 1 gigabyte (float32):
might fit.
To view a high-dimensional tree or graph in 2d,
see Multidimensional Scaling (also from Kruskal),
and the huge literature on dimension reduction.
However, in dim > 20 say, most distances will be near the median,
so I believe dimension reduction cannot work there.
A: I've had good experience with KNIME during one of my project. It 's an excellent solution for quick exploratory mining and graphing. On top of that it provides R and Weka modules seamless integration.  
A: Also have a look at ELKI, an open-source data mining software. Wikimedia commons has a gallery with images produced with ELKI, many of which are related to cluster analysis.
A: GGobi (http://www.ggobi.org/), along with the R package rggobi, is perfectly suited to this task. 
See the related presentation for examples: http://www.ggobi.org/book/2007-infovis/05-clustering.pdf
A: Take a look at Cluster 3.0.  I'm not sure if it will do all you want, but it's pretty well documented and lets you choose from a few distance metrics.  The visualization piece is through a separate program called Java TreeView (screenshot).
A: Weka is an open source program for data mining (wirtten and extensible in Java), Orange is an open source program and library for data mining and machine learning (written in Python). They both allow convenient and efficient visual exploration of multidimensional data
A: GGobi does look interesting for this. Another approach could be to treat your similarity/inverse distance matrices as network adjacency matrices and feeding that into a network analysis routine (e.g., either igraph in R or perhaps Pajek). With this approach I would experiment with cutting the cutting the node distances into a binary tie at various cutpoints.
A: DataMelt free numeric software includes Java library called JMinHep.
 Please look at the manual under the section "Data clustering". It provides a GUI to visualize multidimensional data points in X-Y, and run a number of data clustering algorithms.
