I have several data sets of frequency values (See Fig. 1 for an example).

I'm interested in those tighter clusters (marked by green rectangles) and am using hierarchical clustering in MATLAB (with unweighted average distance method) to separate them. (*)

The spread of these clusters increases with frequency (the standard deviation is positively correlated with the frequency, while the coefficient of variation is not).

So here's my question: Is there a way for the clustering to factor in this relationship, so that the average distance a point has to have to the points of the nearest cluster to be included in that cluster is also dependent on - for example - the mean of that cluster?

I'm thinking that the cutoff would have to be different for each point, but I don't think this is possible in this method. I would also be open to alternative clustering methods, but not k-means, because I don't want to specify the number of clusters in advance.

Also, if you have suggestions on rephrasing my question so that it may be more useful to others I would be grateful.

Thank you for your time!

Frequency cluster example

(*) I'm following this example for the clustering procedure: http://www.mathworks.de/de/help/stats/examples/cluster-analysis.html

  • $\begingroup$ Why not cluster the square roots or logs or reciprocals of the frequencies? (A reciprocal frequency is a wavelength, after all.) If you posit a quantitative model for the relationship between variance and frequency you can use that to derive an appropriate transformation of frequency that will accomplish what you want. What, then, do you know or assume about the relationship between cluster variance and frequency? $\endgroup$ – whuber Mar 11 '14 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.