3
$\begingroup$

On page 599 of this book, the author states (without proving) that for random samples $(X_1, Y_1)$, ..., $(X_n, Y_n)$ from a bivariate normal distribution, the sample correlation coefficient \begin{align} r &= \dfrac{\sum_{i=1}^n (X_i - \bar X )(Y_i - \bar Y)}{\sqrt{\sum_{i=1}^n (X_i - \bar X )^2 \sum_{i=1}^n(Y_i - \bar Y)^2}} \\ & = \dfrac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} \end{align} is also the maximum-likelihood estimator of $\rho$, the correlation coefficient.

However I don't know how to prove it. For bivariate distribution $(X, Y)$, we have pdf $$f(x,y) = \frac{1}{2\pi \sqrt{1-\rho^2} \sigma_x \sigma_y} \exp(-\frac{1}{2(1-\rho^2)}[(\frac{x-\mu_x}{\sigma_x})^2 - 2 \rho (\frac{x-\mu_x}{\sigma_x})(\frac{y-\mu_y}{\sigma_y})+(\frac{y-\mu_y}{\sigma_y})^2]).$$

The log-likelihood would yield $$\ln L = -n \ln (2\pi \sqrt{1-\rho^2} \sigma_x \sigma_y) + \sum -\frac{1}{2(1-\rho^2)}[...]$$ and taking the partial derivative w.r.t. $\rho$ and setting it to be zero doesn't seem to take me anywhere.

I have found some good resources for special case when $\mu_x = \mu_y = 0$ and $\mathbb{E}[X_i^2] = \mathbb{E}[Y_i^2] = 1$, but I am wondering how to approach the general problem, or if someone can direct me to a great resource I would really appreciate it.

$\endgroup$
7
$\begingroup$

Given two random variables $X$ and $Y$, their correlation coefficient is:

$$\rho_{XY} = \frac{Cov(X,Y)}{\sqrt{Var(X)\cdot Var(Y)}}$$

Where $Cov(X,Y)$ is the covariance of $X$ and $Y$, $Var(X)$ is the variance of $X$, and $Var(Y)$ is the variance of $Y$.

According to your book, the maximum likelihood estimator of $\rho_{XY}$ is:

\begin{align} r &= \dfrac{\sum_{i=1}^n (X_i - \bar X )(Y_i - \bar Y)}{\sqrt{\sum_{i=1}^n (X_i - \bar X )^2 \sum_{i=1}^n(Y_i - \bar Y)^2}} \\ & = \dfrac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} \end{align}

Notice that:

$$\frac{1}{n} \sum_{i=1}^n (X_i - \bar X )(Y_i - \bar Y)$$

is the maximum likelihood estimator of the covariance of $X$ and $Y$. Also, notice that:

$$\frac{1}{n} \sum_{i=1}^n (X_i - \bar X )^2$$

is the (biased) maximum likelihood estimator of the variance of $X$ and:

$$\frac{1}{n} \sum_{i=1}^n(Y_i - \bar Y)^2$$

is the (biased) maximum likelihood estimator of the variance of $Y$.

So, the (biased) maximum likelihood estimator of the correlation coefficient is equal to:

$$r = \frac{\frac{1}{n} \sum_{i=1}^n (X_i - \bar X )(Y_i - \bar Y)}{\sqrt{\frac{1}{n} \sum_{i=1}^n (X_i - \bar X )^2 \cdot \frac{1}{n} \sum_{i=1}^n(Y_i - \bar Y)^2}}$$

The $\frac{1}{n}$ term is multiplied through in the denominator, so:

$$r = \frac{\frac{1}{n} \sum_{i=1}^n (X_i - \bar X )(Y_i - \bar Y)}{\frac{1}{n} \sqrt{\sum_{i=1}^n (X_i - \bar X )^2 \cdot \sum_{i=1}^n(Y_i - \bar Y)^2}} = \frac{\sum_{i=1}^n (X_i - \bar X )(Y_i - \bar Y)}{\sqrt{\sum_{i=1}^n (X_i - \bar X )^2 \cdot \sum_{i=1}^n(Y_i - \bar Y)^2}}$$

Note that this follows from the fact that a function of maximum likelihood estimators is a maximum likelihood estimator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.