2
$\begingroup$

I am performing K means clustering on a gene expression dataset. I am aware of the fact that the Pearson correlation metric allows to group trends or patterns irrespective of their overall level of expression. I was wondering if the same concept stands for Covariance metric (I believe that the only difference between the two metrics is the fact that covariance returns unbounded values, while Pearson maps value in interval [-1,1])

$\endgroup$

2 Answers 2

0
$\begingroup$

If you look up the definitions of Pearson, one of them is

$$ \rho := \frac{\text{Cov}(X,Y)}{\sigma_X\sigma_Y} $$

which obviously is a standardization of the covariance. Data with unit variance, they are identical.

But what actually was your question?

$\endgroup$
4
  • $\begingroup$ Thank you for your answer. My question regarded specifically the impact of the variables' absolute values on clustering results. From what you are saying, the absence of standardization in covariance metric should matter. For the Pearson metric it is otherwise clear that the standardization reduces the impact of relevant differences, and it is suitable for the observations of trends. Is this correct? $\endgroup$
    – Alberto
    Nov 14, 2012 at 12:28
  • $\begingroup$ I do not get the full picture of your question, as k-means is designed for Euclidean distance. Using it with e.g. pearson or covariance may in the worst case prevent k-means from converging. $\endgroup$ Nov 14, 2012 at 15:08
  • $\begingroup$ @Anony-Mousse k-means works with any Bregman divergence jmlr.org/papers/volume6/banerjee05b/banerjee05b.pdf (Banerjee, 2005) not only Euclidean (a special case). By the way, k-means can viewed as the discrete solution of PCA ranger.uta.edu/~chqding/papers/KmeansPCA1.pdf (Ding, 2004) which works with covariance matrices. Correlation / Covariances is very $L_2$ flavoured for the random variables : $\mathbb{E}[(X-Y)^2] = (\mu_X - \mu_Y)^2 + (\sigma_X - \sigma_Y)^2 + 2\sigma_X \sigma_Y (1-\rho(X,Y))$. $\endgroup$
    – mic
    Oct 10, 2015 at 20:10
  • 1
    $\begingroup$ I doubt pearson correlation is a Bregman divergence. In particular, it is not defined on a 2d point such as (0,0) or (1,1) and these may arise during k-means even if they weren't in your original data. $\endgroup$ Oct 10, 2015 at 20:35
0
$\begingroup$

k-means cluster analysis uses the variance and not the correlation. Many people "sphere" data prior to applying k-means (i.e., subtract the mean from each variable and then divide each variable by its standard deviation). This prevents variables with large variances from dominating the k-means solution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.