Clustering distributions

Question

I have several distributions (10 distributions in the figure below).

In fact these are histograms: there are 70 values on the x-axis which are the sizes of some particles in a solution and for each value of x the corresponding value of y is the proportion of particles whose size is around the value of x.

I would like to cluster these distributions. Currently I use a hierarchical clustering with the Euclidean distance for example. I am not satisfied by the choice of the distance. I have tried information-theoretic distance such as Kullback-Leibler but there are many zeros in the data and this causes difficulties. Do you have a proposal of an appropriate distance and/or another clustering method ?

Answer 1

I understand you such that all distributions can potentially take on the same 70 discrete values. Then it will be easy for you to compare cumulative curves of the distributions (comparing cumulative curves is the general way to compare distributions). That will be omnibus comparison for differences in shape, location, and spread.

So, prepare data in the form like (A, B, ... etc are the distributions)

Value CumProp_A CumProp_B ...
1       .01       .05
2       .12       .14
...     ...       ...
70      1.00      1.00

and compute a distance matrix between the distributions. Submit to hierarchical clustering (I'd recommend complete linkage method). What distance? Well, if you think two cumulative curves are very different if they are far apart just at one value (b), use Chebyshev distance. If you think two cumulative curves are very different only if one is stably above the other along a wide range of values (c), use autocorrelative distance. In case any local differences between the curves are important (a), use Manhattan distance.

P.S. Autocorrelative distance is just a non-normalized coefficient of autocorrelation of differences between the cumulative curves X and Y:

$\sum_{i=2}^N (X-Y)_i*(X-Y)_{i-1}$

Answer 2

If your data are histograms, you might want to look into appropriat distance functions for that such as the "histogram intersection distance".

There is a tool called ELKI that has a wide variety of clustering algorithms (much more modern ones than k-means and hierarchical clustering) and it even has a version of histogram intersection distance included, that you can use in most algorithms. You might want to try out a few of the algorithms available in it. From the plot you gave above, it is unclear to me what you want to do. Group the individual histograms, right? Judging from the 10 you showed above, there might be no clusters.

Answer 3

You may want to use some feature extraction technique to derive descriptors for a k-means or other type of clustering.

A basic approach would be to fit a certain distribution to your histograms and use its parameters as descriptors. For instance, you seem to have bimodal distributions, that you can describe with 2 means and 2 standard deviations.

Another possibility is to cluster over the first two or three principal component of the counts of the histograms.

Alternatively wavelets approaches can be used.

This page explains how to do that when dealing with extracellular spikes. The data is different, but the idea should be applicable to your case. You will also find many references at the bottom.

http://www.scholarpedia.org/article/Spike_sorting

In R you can calculate the principal components of your peaks using either the princomp or prcomp function. Here you'll find a tutorial on PCA in R.

For wavelets you may look at the wavelets package.

k-means clustering can be achieved using the kmeans function.