# Cramer-Rao Lower Bound for the estimation of Pearson correlation

Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ when estimating $\rho$ on finite sample with the Pearson estimator.

Is there any known Cramer-Rao lower bound for that?

## 2 Answers

Yes, there is and it can be derived routinely. The Fisher Information can be shown to be

$$I(\rho) = \frac{1+\rho^2}{\left(1-\rho^2\right)^2}$$

and you know how to get the CRLB from here. The result may be arrived at simply by applying the definition of Fisher Information, i.e. start from the log likelihood

$$\log\left[ f(x;\rho) \right] =\log\left\{ \frac{1}{2\pi \sqrt{1-\rho^2}}\exp\left\{-\frac{1}{2(1-\rho^2)} \left(x^2 + y^2 - 2\rho xy \right) \right\} \right\}$$

take the derivatives and evaluate the expectation using the properties of the normal distribution. I advise you to verify it on your own.

• Thank you for the precise answer. Any chance you know theoretical results involving the sample size? I'm looking for pointers to the notions. For example, what can I say if my sample size is 250, 500, ...? – mic Feb 14 '16 at 20:34
• Sample sizes for what? What precisely are you trying to accomplish? – JohnK Feb 14 '16 at 20:36
• I draw $N$ realizations from a bivariate gaussian with correlation $\rho$, then I would like to know how far $\hat{\rho}$ deviates from $\rho$, some sort of sharp concentration bounds. – mic Feb 14 '16 at 20:46
• Do you know if Pearson estimator for linear correlation achieves its Cramer-Rao lower bound? I do not have find any document yet on it. – mic Feb 21 '16 at 11:34

I did the computations on my own, but I find something different:

We consider the set of $2 \times 2$ correlation matrices $C = \begin{pmatrix} 1 & \theta \\ \theta & 1 \end{pmatrix}$ parameterized by $\theta$.

Let $x = \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \in \mathbf{R}^2$.

$f(x;\theta) = \frac{1}{2\pi \sqrt{1-\theta^2}} \exp\left(-\frac{1}{2}x^\top C^{-1} x \right) = \frac{1}{2\pi \sqrt{1-\theta^2}} \exp\left( -\frac{1}{2(1-\theta^2)}(x_1^2 + x_2^2 - 2\theta x_1 x_2) \right)$

$\log f(x;\theta) = - \log(2\pi \sqrt{1-\theta^2}) -\frac{1}{2(1-\theta^2)}(x_1^2 + x_2^2 - 2\theta x_1 x_2)$

$\frac{\partial^2 \log f(x;\theta)}{\partial \theta^2} = -\frac{\theta^2 + 1}{(\theta^2 - 1)^2} - \frac{x_1^2}{2(\theta+1)^3} + \frac{x_1^2}{2(\theta-1)^3} - \frac{x_2^2}{2(\theta+1)^3} + \frac{x_2^2}{2(\theta-1)^3} - \frac{x_1 x_2}{(\theta+1)^3} - \frac{x_1 x_2}{(\theta-1)^3}$

Then, we compute $\int_{-\infty}^{\infty} \frac{\partial^2 \log f(x;\theta)}{\partial \theta^2} f(x;\theta) dx$. Since $\mathbf{E}[x_1] = \mathbf{E}[x_2] = 0$, $\mathbf{E}[x_1x_2] = \theta$, $\mathbf{E}[x_1^2] = \mathbf{E}[x_2^2] = 1$, we get

$\int_{-\infty}^{\infty} \frac{\partial^2 \log f(x;\theta)}{\partial \theta^2} f(x;\theta) dx = - \frac{\theta^2 + 1}{(\theta^2 - 1)^2} - \frac{1}{2(\theta+1)^3} + \frac{1}{2(\theta-1)^3} - \frac{1}{2(\theta+1)^3} + \frac{1}{2(\theta-1)^3} - \frac{\theta}{(\theta+1)^3} - \frac{\theta}{(\theta-1)^3} = - \frac{3(\theta^2+1)}{(\theta-1)^2(\theta+1)^2}$

Thus, $$g(\theta) = \frac{3(\theta^2+1)}{(\theta-1)^2(\theta+1)^2}.$$