Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 7 down vote favorite
2

Various forms of the correlation, e.g., $r = \frac{\Sigma_i x_i * y_i}{\sigma_x \sigma_y}$ or $r = \frac{\Sigma_i (x_i-\bar{x}) * (y_i-\bar{y})}{\sigma_x \sigma_y}$ are popular similarity measures in many applications.

Is there a probabilistic interpretation for this such that either $r$ or $r^2$ is an approximate likelihood for x and y coming from the same or similar distribution? i.e., if we have some form of $P_{\theta_1}(x)$ and $P_{\theta_2}(y)$, then $r$ is related to $P(\theta_1=\theta_2 | x,y)$?

I would guess that the correlation may be the first term in the approximation of some sort of a likelihood measure. But I am unable to arrive at such a model. Assuming $x$ and $y$ to come from a normal, and $\theta$ being the mean, it doesn't really derive that expression.

share|improve this question
1  
The "similarity" indexed by correlation refers to the degree to which high/low values in one distribution (relative to that distribution's mean and variance) are paired with high/low values from a second distribution (relative to that second distribution's mean and variance). Since high/low are relative, the two distributions can be very dissimilar in both mean and variance and still obtain a high correlation, so I don't think you can use the correlation as a means of comparing distributions. – Mike Lawrence May 10 '11 at 15:22
@Mike, What if I let the scaling vary. Suppose that $x$ and $y$ are multivariate Gaussians, with a simple diagonal covariance. And then, I just test for the two means (which are vectors) are linearly related. i.e., $\mu_1 = s \mu_2$ for any scalar $s$? – highBandWidth May 10 '11 at 15:24

2 Answers

up vote 1 down vote

I think there's a slight conceptual problem here: the likelihood is the quantity $P(x,y|\theta)$ as a function of $\theta$. But the correlation coefficient is a function of the data $x$ and $y$. The correlation coefficient is therefore by definition a "statistic" (a function of the data) and not a likelihood (a function of the parameters).

To put it another way, there are values of $\theta$ for which the probability of the observed correlation would be high, and other values of $\theta$ for which the observed correlation would be low. If you have a correlated Gaussian in mind, then the observed correlation might be unlikely due to the fact that the covariance matrix was either more dependent or less dependent than would be expected from the data. (So the relationship between likelihood and $r^2$ will not even be monotonic unless $r^2=1$). Of course, the expected correlation coefficient is monotonically related to the off-diagonal term in a bivariate Gaussian, but this doesn't make $r^2$ itself a proxy for likelihood.

It sounds like you're interested in a test of whether a particular correlation coefficient is likely to have been observed given that $x$ and $y$ were generated independently. For this, you might want to perform a significance test on the value of the observed $r$ or $r^2$, which will tell you how probable it is that you observed that value given a particular model of the data. The Wikipedia article on correlation coefficient inference discusses a few common tests.

share|improve this answer
up vote 0 down vote accepted

I think PCA does something close to a Guassian noise model that comes close to a correlation measure in a multidimensional space.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.