Given a bivariate Gaussian distribution $\mathcal{N}\left(0,\begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$, I am looking for information on the distribution of $\hat{\rho}$ when estimating $\rho$ on finite sample with the Pearson estimator.

Is there any known Cramer-Rao lower bound for that?


Yes, there is and it can be derived routinely. The Fisher Information can be shown to be

$$I(\rho) = \frac{1+\rho^2}{\left(1-\rho^2\right)^2}$$

and you know how to get the CRLB from here. The result may be arrived at simply by applying the definition of Fisher Information, i.e. start from the log likelihood

$$\log\left[ f(x;\rho) \right] =\log\left\{ \frac{1}{2\pi \sqrt{1-\rho^2}}\exp\left\{-\frac{1}{2(1-\rho^2)} \left(x^2 + y^2 - 2\rho xy \right) \right\} \right\}$$

take the derivatives and evaluate the expectation using the properties of the normal distribution. I advise you to verify it on your own.

  • $\begingroup$ Thank you for the precise answer. Any chance you know theoretical results involving the sample size? I'm looking for pointers to the notions. For example, what can I say if my sample size is 250, 500, ...? $\endgroup$ – mic Feb 14 '16 at 20:34
  • $\begingroup$ Sample sizes for what? What precisely are you trying to accomplish? $\endgroup$ – JohnK Feb 14 '16 at 20:36
  • $\begingroup$ I draw $N$ realizations from a bivariate gaussian with correlation $\rho$, then I would like to know how far $\hat{\rho}$ deviates from $\rho$, some sort of sharp concentration bounds. $\endgroup$ – mic Feb 14 '16 at 20:46
  • $\begingroup$ Do you know if Pearson estimator for linear correlation achieves its Cramer-Rao lower bound? I do not have find any document yet on it. $\endgroup$ – mic Feb 21 '16 at 11:34

I did the computations on my own, but I find something different:

We consider the set of $2 \times 2$ correlation matrices $C = \begin{pmatrix} 1 & \theta \\ \theta & 1 \end{pmatrix}$ parameterized by $\theta$.

Let $x = \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \in \mathbf{R}^2$.

$f(x;\theta) = \frac{1}{2\pi \sqrt{1-\theta^2}} \exp\left(-\frac{1}{2}x^\top C^{-1} x \right) = \frac{1}{2\pi \sqrt{1-\theta^2}} \exp\left( -\frac{1}{2(1-\theta^2)}(x_1^2 + x_2^2 - 2\theta x_1 x_2) \right)$

$\log f(x;\theta) = - \log(2\pi \sqrt{1-\theta^2}) -\frac{1}{2(1-\theta^2)}(x_1^2 + x_2^2 - 2\theta x_1 x_2) $

$\frac{\partial^2 \log f(x;\theta)}{\partial \theta^2} = -\frac{\theta^2 + 1}{(\theta^2 - 1)^2} - \frac{x_1^2}{2(\theta+1)^3} + \frac{x_1^2}{2(\theta-1)^3} - \frac{x_2^2}{2(\theta+1)^3} + \frac{x_2^2}{2(\theta-1)^3} - \frac{x_1 x_2}{(\theta+1)^3} - \frac{x_1 x_2}{(\theta-1)^3} $

Then, we compute $\int_{-\infty}^{\infty} \frac{\partial^2 \log f(x;\theta)}{\partial \theta^2} f(x;\theta) dx$. Since $\mathbf{E}[x_1] = \mathbf{E}[x_2] = 0$, $\mathbf{E}[x_1x_2] = \theta$, $\mathbf{E}[x_1^2] = \mathbf{E}[x_2^2] = 1$, we get

$\int_{-\infty}^{\infty} \frac{\partial^2 \log f(x;\theta)}{\partial \theta^2} f(x;\theta) dx = - \frac{\theta^2 + 1}{(\theta^2 - 1)^2} - \frac{1}{2(\theta+1)^3} + \frac{1}{2(\theta-1)^3} - \frac{1}{2(\theta+1)^3} + \frac{1}{2(\theta-1)^3} - \frac{\theta}{(\theta+1)^3} - \frac{\theta}{(\theta-1)^3} = - \frac{3(\theta^2+1)}{(\theta-1)^2(\theta+1)^2} $

Thus, $$g(\theta) = \frac{3(\theta^2+1)}{(\theta-1)^2(\theta+1)^2}.$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.