I am looking to find the finest bound possible for $$\mathbb{P}\left( | \hat{\rho}(X,Y) - \rho(X,Y) | \geq t\right) \leq ?$$

where $\rho$ is the Pearson correlation defined as

$$\rho(X,Y) = \frac{\mathbb{E}[XY] - \mathbb{E}[X] \mathbb{E}[Y]}{\sqrt{\mathbb{E}[X^2] - \mathbb{E}[X]^2}\sqrt{\mathbb{E}[Y^2] - \mathbb{E}[Y]^2}} $$

and its statistical estimate

$$\hat{\rho}(X,Y) = \frac{\sum_{t=1}^T (X^t - \overline{X})(Y^t - \overline{Y})}{\sqrt{\sum_{t=1}^T \left(X^t - \overline{X}\right)^2}\sqrt{\sum_{t=1}^T \left(Y^t - \overline{Y}\right)^2}}$$

I think Fisher may have worked on it, but I did not find any relevant paper.


I derived the following bound, not sure if it's very sharp.

Here is the baseline: $X_i \sim \mathcal{N}(0,1)$, their correlation is $\rho$. I transform the squared sum of Gaussian differences into squared sum of independent Gaussians ($\Delta_ij \sim 2(1-\rho)\mathcal{N}(0,1)$) which follow a $\chi^2(T)$. I then apply a $\chi^2$ tail inequality to obtain the result.

Hope it is correct.

\begin{eqnarray} \mathbb{P}\left( \left|\hat{\rho}(X_i,X_j) - \rho(X_i,X_j) \right| \geq t \right) & = & \mathbb{P}\left( \left|\hat{\rho}(X_i,X_j) - \mathbb{E}\left[\hat{\rho}(X_i,X_j)\right] \right| \geq t \right) \\ & = & \mathbb{P}\left( \left| 1 - \frac{1}{2T}\sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 - 1 + \frac{1}{2T}\mathbb{E}\left[ \sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 \right] \right| \geq t \right) \\ & = & \mathbb{P}\left( \left| \sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 - \mathbb{E}\left[ \sum_{t=1}^T \left(X_i^t - X_j^t \right)^2 \right] \right| \geq 2Tt \right) \\ & = & \mathbb{P}\left( 2(1-\rho) \left| \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \right| \geq 2Tt \right) \\ & = & \mathbb{P}\left( \left| \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \right| \geq \frac{Tt}{1-\rho} \right) \end{eqnarray}

To conclude we use the following lemma and its corolary

Generalized exponential inequality for chi-square distributions (p. 1365)

Let $(Y_1,\ldots,Y_D)$ be i.i.d. Gaussian variables, with mean 0 and variance 1. Let $a_1,\ldots,a_D$ be nonnegative. We set $|a|_\infty = \sup_i |a_i|$ and $|a|_2^2 = \sum_{i=1}^D a_i^2$. Let $Z = \sum_{i=1}^D a_i(Y_i^2 - 1)$. Then, the following inequalities hold for any positive $x$: $$\mathbb{P}(Z \geq 2|a|_2 \sqrt{x} + 2|a|_\infty x) \leq \exp(-x)$$ $$\mathbb{P}(Z \leq -2|a|_2 \sqrt{x}) \leq \exp(-x)$$

As an immediate corrolary we obtain Exponential inequality for chi-square distributions

Let $U$ be a $\chi^2$ with $D$ degrees of freedom. For any positive $x$, $$\mathbb{P}(U-D \geq 2 \sqrt{Tx} + 2x) \leq \exp(-x)$$ $$\mathbb{P}(D-U \geq 2 \sqrt{Tx}) \leq \exp(-x)$$

We obtain

$$\mathbb{P}\left( \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \geq \frac{Tt}{1-\rho} \right) \leq \exp\left\{ -\frac{1}{2}T \left(\sqrt{1 + \frac{2t}{1-\rho}} + 1 + \frac{t}{1-\rho} \right) \right\}$$ and $$\mathbb{P}\left( \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] - \sum_{t=1}^T {\Delta_{ij}^t}^2 \geq \frac{Tt}{1-\rho} \right) \leq \exp\left\{ -\frac{1}{4}T \frac{t^2}{(1-\rho)^2} \right\}$$


$$\mathbb{P}\left( \left| \sum_{t=1}^T {\Delta_{ij}^t}^2 - \mathbb{E}\left[ \sum_{t=1}^T {\Delta_{ij}^t}^2 \right] \right| \geq \frac{Tt}{1-\rho} \right) \leq \exp\left\{ -\frac{1}{4}T \frac{t^2}{(1-\rho)^2} \right\} + \exp\left\{ -\frac{1}{2}T \left(\sqrt{1 + \frac{2t}{1-\rho}} + 1 + \frac{t}{1-\rho} \right) \right\}$$


$$\mathbb{P}\left( \left|\hat{\rho}(X_i,X_j) - \rho(X_i,X_j) \right| \geq t \right) \leq \exp\left\{ -\frac{1}{4}T \frac{t^2}{(1-\rho)^2} \right\} + \exp\left\{ -\frac{1}{2}T \left(\sqrt{1 + \frac{2t}{1-\rho}} + 1 + \frac{t}{1-\rho} \right) \right\}$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.