# convergence rate of Pearson correlation

I am looking to find the finest bound possible for $$\mathbb{P}\left( | \hat{\rho}(X,Y) - \rho(X,Y) | \geq t\right) \leq ?$$

where $\rho$ is the Pearson correlation defined as

$$\rho(X,Y) = \frac{\mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y]}{\sqrt{\mathbb{E}[X^2] - \mathbb{E}[X]^2}\sqrt{\mathbb{E}[Y^2] - \mathbb{E}[Y]^2}}$$

and its statistical estimate

$$\hat{\rho}(X,Y) = \frac{\sum_{t=1}^T (X^t - \overline{X})(Y^t - \overline{Y})}{\sqrt{\sum_{t=1}^T \left(X^t - \overline{X}\right)^2}\sqrt{\sum_{t=1}^T \left(Y^t - \overline{Y}\right)^2}}$$

I think Fisher may have worked on it, but I did not find any relevant paper.

I derived the following bound, not sure if it's very sharp.

Here is the baseline: $X_i \sim \mathcal{N}(0,1)$, their correlation is $\rho$. I transform the squared sum of Gaussian differences into squared sum of independent Gaussians ($\Delta_ij \sim 2(1-\rho)\mathcal{N}(0,1)$) which follow a $\chi^2(T)$. I then apply a $\chi^2$ tail inequality to obtain the result.

Hope it is correct.

\begin{eqnarray} \mathbb{P}\left( \left|\hat{\rho}(X_i,X_j) - \rho(X_i,X_j) \right| \geq t \right) & = & \mathbb{P}\left( \left|\hat{\rho}(X_i,X_j) - \mathbb{E}\left[\hat{\rho}(X_i,X_j)\right] \right| \geq t \right) \\ & = & \mathbb{P}\left( \left| 1 - \frac{1}{2T}\sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 - 1 + \frac{1}{2T}\mathbb{E}\left[ \sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 \right] \right| \geq t \right) \\ & = & \mathbb{P}\left( \left| \sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 - \mathbb{E}\left[ \sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 \right] \right| \geq 2Tt \right) \\ & = & \mathbb{P}\left( 2(1-\rho) \left| \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \right| \geq 2Tt \right) \\ & = & \mathbb{P}\left( \left| \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \right| \geq \frac{Tt}{1-\rho} \right) \end{eqnarray}

To conclude we use the following lemma and its corolary

Let $(Y_1,\ldots,Y_D)$ be i.i.d. Gaussian variables, with mean 0 and variance 1. Let $a_1,\ldots,a_D$ be nonnegative. We set $|a|_\infty = \sup_i |a_i|$ and $|a|_2^2 = \sum_{i=1}^D a_i^2$. Let $Z = \sum_{i=1}^D a_i(Y_i^2 - 1)$. Then, the following inequalities hold for any positive $x$: $$\mathbb{P}(Z \geq 2|a|_2 \sqrt{x} + 2|a|_\infty x) \leq \exp(-x)$$ $$\mathbb{P}(Z \leq -2|a|_2 \sqrt{x}) \leq \exp(-x)$$

As an immediate corrolary we obtain Exponential inequality for chi-square distributions

Let $U$ be a $\chi^2$ with $D$ degrees of freedom. For any positive $x$, $$\mathbb{P}(U-D \geq 2 \sqrt{Tx} + 2x) \leq \exp(-x)$$ $$\mathbb{P}(D-U \geq 2 \sqrt{Tx}) \leq \exp(-x)$$

We obtain

$$\mathbb{P}\left( \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \geq \frac{Tt}{1-\rho} \right) \leq \exp\left\{ -\frac{1}{2}T \left(\sqrt{1 + \frac{2t}{1-\rho}} + 1 + \frac{t}{1-\rho} \right) \right\}$$ and $$\mathbb{P}\left( \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] - \sum_{t=1}^T {\Delta_{ij}^t}^2 \geq \frac{Tt}{1-\rho} \right) \leq \exp\left\{ -\frac{1}{4}T \frac{t^2}{(1-\rho)^2} \right\}$$

Therefore

$$\mathbb{P}\left( \left| \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \right| \geq \frac{Tt}{1-\rho} \right) \leq \exp\left\{ -\frac{1}{4}T \frac{t^2}{(1-\rho)^2} \right\} + \exp\left\{ -\frac{1}{2}T \left(\sqrt{1 + \frac{2t}{1-\rho}} + 1 + \frac{t}{1-\rho} \right) \right\}$$

Finally,

$$\mathbb{P}\left( \left|\hat{\rho}(X_i,X_j) - \rho(X_i,X_j) \right| \geq t \right) \leq \exp\left\{ -\frac{1}{4}T \frac{t^2}{(1-\rho)^2} \right\} + \exp\left\{ -\frac{1}{2}T \left(\sqrt{1 + \frac{2t}{1-\rho}} + 1 + \frac{t}{1-\rho} \right) \right\}$$