1
$\begingroup$

I'm doing clustering on data with 10 dimensional data. I recently had to revaluate my original cluster analysis because I found (by accident) that some of the variables are closely correlated. I assume that using any kind of linear distance measure, such as Euclidean, with correlated variables essentially adds increasing weight to the underlying component for every correlated variable that is used. This therefore could generate biases in the distance measures that I would not necessarily be aware of.

Are there any distance measures or clustering methods that account for this? Assuming I might have 1000 dimensional data then it would be hard to do manual examination of the correlations. Off the top of my head, I guess I could reduce down to the principle components that explain 90% of the variation in the data, then do the clustering and distance measures on these components?

I've attached a PCA plot with the loadings to provide an example of the correlated variables. Happy to recieve any answers that point out if I'm making the XY mistake.

enter image description here

$\endgroup$
  • $\begingroup$ I assume that using any kind of linear distance measure, such as Euclidean, with correlated variables essentially adds increasing weight to... It is all enigmatic. Euclidean distance will be approximately the same in 2d (2-variable space) and in 1d (1st component space) if the variables are stroggly correlated (i.e. the data cloud is diagonal sausage). Variable correlations is not a general, universal obstacle for doing a cluster analysis. $\endgroup$ – ttnphns Apr 5 '17 at 8:22
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/62092/… $\endgroup$ – kjetil b halvorsen Aug 29 '17 at 13:54
1
$\begingroup$

I would start by reading the question, How would PCA help with a k-means clustering analysis?, which suggests doing the sort of PCA in your question.

$\endgroup$
0
$\begingroup$

If you want to remove correlations and have the same weight on each component, this has a name: **Principal Component Analysis*.

Clearly, it removes all correlations.

But it does not entirely solve the weighting problem. It will put too much weight on minor components.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.