Best Clustering Technique for Probability Scores I have a data which which have 17 variables i.e. 17-Dimension data. The Data is a result of Max-Diff exercise which is performed for ranking these 17 attributes and have comparative preference/importance and probability of choice of each
attribute. So in summary this data is of around 2000 rows and 17 columns values ranging between 0 and 1. Now I want to create meaningful market segments using this data. Just wondering if anyone can help me in deciding best clustering technique for completing this task.
Any suggestion will be highly appreciated. Thanks in advance.
 A: If your data are 17-dimensional probability histograms (i.e. every row sums up to 1) then you can try the following:


*

*Format your data as CSV, 2000 rows, 17 columns (+ a textual label column if you have one)

*Load the data into ELKI

*Choose a hierarchy extraction algorithm from the clustering.hierarchical.extraction (or so) package. ''Update'': for example clustering.hierarchical.extraction.SimplifiedHierarchyExtraction which is a nice automatic way for cutting the tree without having to choose the height or the number of clusters k.

*Choose Anderberg hierarchical clustering: AnderbergHierarchicalClustering because it is faster than AGNES.

*Try complete linkage first and maybe GroupAverageLinkageMethod

*Choose a distance function from the probability package, say chi squared distance probabilistic.ChiSquaredDistanceFunction or probabilistic.SqrtJensenShannonDivergenceDistanceFunction (these worked well for me on such data)


Then look at the dendrogram, if this exhibits nice clusters. There probably are some outliers, too. I'd focus on the dendrogram, and only treat the automatic extraction (step 3) as a guide. You have several parameters to vary to get a good result. But at the end, you really need to look at the data again, if it is any good for your problem.
Here is an example dendrogram, clusters extracted with above simplified extraction and minimum size 50 objects. The data are color histograms, Chi^2 distance, group average linkage.

From that plot, I would try again with a larger minimum cluster size.
Let's try sqrt jensen shannon, complete linkage, and min size 100:

The yellow cluster to the right should probably be cut into two (probably one is less than 100 objects). Some clusters have an outlier that should be dropped. But maybe not too bad for a start.
Too bad you can't "edit" the extracted clusters in the plot. I'm not aware of an easy way to identify e.g. an outlier here easily.
