Correlation as a likelihood measure Various forms of the correlation, e.g.,
$r =  \frac{\Sigma_i x_i * y_i}{\sigma_x  \sigma_y}$
or
$r = \frac{\Sigma_i (x_i-\bar{x}) * (y_i-\bar{y})}{\sigma_x  \sigma_y}$
are popular similarity measures in many applications.
Is there a probabilistic interpretation for this such that either $r$ or $r^2$ is an approximate likelihood for x and y coming from the same or similar distribution? i.e., if we have some form of $P_{\theta_1}(x)$ and $P_{\theta_2}(y)$, then $r$ is related to $P(\theta_1=\theta_2 | x,y)$?
I would guess that the correlation may be the first term in the approximation of some sort of a likelihood measure. But I am unable to arrive at such a model. Assuming $x$ and $y$ to come from a normal, and $\theta$ being the mean, it doesn't really derive that expression.
 A: I think there's a slight conceptual problem here: the likelihood is the quantity $P(x,y|\theta)$ as a function of $\theta$. But the correlation coefficient is a function of the data $x$ and $y$. The correlation coefficient is therefore by definition a "statistic" (a function of the data) and not a likelihood (a function of the parameters).  
To put it another way, there are values of $\theta$ for which the probability of the observed correlation would be high, and other values of $\theta$ for which the observed correlation would be low. If you have a correlated Gaussian in mind, then the observed correlation might be unlikely due to the fact that the covariance matrix was either more dependent or less dependent than would be expected from the data.  (So the relationship between likelihood and $r^2$ will not even be monotonic unless $r^2=1$).  Of course, the expected correlation coefficient is monotonically related to the off-diagonal term in a bivariate Gaussian, but this doesn't make $r^2$ itself a proxy for likelihood.  
It sounds like you're interested in a test of whether a particular correlation coefficient is likely to have been observed given that $x$ and $y$ were generated independently.  For this, you might want to perform a significance test on the value of the observed $r$ or $r^2$, which will tell you how probable it is that you observed that value given a particular model of the data.  The Wikipedia article on correlation coefficient inference discusses a few common tests.
A: I think PCA does something close to a Guassian noise model that comes close to a correlation measure in a multidimensional space.
