# Statistics of sample correlation for uniformly distributed samples

I am computing the sample correlation between two vectors of uncorrelated and uniformly distributed samples using MATLAB. More precisely, I compute $$r_N=\frac{1}{N}\sum_{i=1}^N x_{i}\, y_{i},$$ where $E(X)=E(Y)={\rm cov}(X,Y)=0$, and $x_{i}, y_{i}$ are drawn from a uniform distribution in $[-A/2,A/2]$. It is clear that $\lim_{N \rightarrow \infty} r_N=0$. However, I would like to compute an estimate of $\tilde{N}$ such that, given $\epsilon >0$ and $\alpha >0$, we have ${\rm Pr}(|r_N| < \epsilon) < \alpha$ for all $N > \tilde{N}$.

In other words, I need an expression for the sample statistic $r_N$. I formulated this question for uniformly distributed data; however I'm interested to know what happens for other distributions (say normal).

Any ideas?

• Since $x$ and $y$ have zero mean $r_N$ has the same expression of $r_{xy}$ in http://en.wikipedia.org/wiki/Correlation minus the normalization by the sample standard deviations so I'm not sure about why you don't see the correlation @COOLSerdash. Commented Jul 12, 2013 at 22:52
• You're looking at sampling error. Watch what happens when you increase $N$ to $10^6$, then $10^7$, then $10^8$, etc.
– whuber
Commented Jul 13, 2013 at 2:26
• I see your point @whuber. So my question now is: is there a way to estimate $N$ such that $P(\left|r_N\right|<\epsilon)>p$ ? Commented Jul 13, 2013 at 16:19
• It's not clear that $\lim_{N\to\infty} r_N = 0$. Indeed, $r_N$ is an estimator of ${\rm E}(XY)$, and ${\rm E}(XY) = {\rm cov}(X, Y)$ since ${\rm E}(X)={\rm E}(X)=0$. So, if ${\rm cov}(X, Y) \neq 0$ then $\lim_{N\to\infty} r_N \neq 0$. This implies that $\tilde{N}$ might not exist. Is the covariance/correlation between $X$ and $Y$ supposed to be zero? Commented Jul 16, 2013 at 23:30
• Yes, I assume that ${\rm cov}(X,Y)=0$ so I added that to the question. Commented Jul 17, 2013 at 16:19

Your question is indeed asking for the finite sample distribution of $r_{N}$. To address your question, let me rephrase it in terms of linear regressions. So a linkage between $r_{N}$ and the ordinary least square (OLS) estimator could be highlighted. You observe two variables $\left\{ x_{i}\right\} _{i=1}^{N}$ and $\left\{ y_{i}\right\} _{i=1}^{N}$, and they are uncorrelated. Suppose you run a linear regression between $y_{i}$ and $x_{i}$, $$y_{i}=\beta x_{i}+u_{i},\quad i=1,\ldots,N.$$ Since $y_{i}$ and $x_{i}$ are uncorrelated, the true value $\beta$ is $0$. Note that the OLS estimator of $\beta$ is $$\hat{\beta}_{N}=\frac{\sum_{i=1}^{N}x_{i}y_{i}}{\sum_{i=1}^{N}x_{i}x_{i}}=\frac{N^{-1}\sum_{i=1}^{N}x_{i}y_{i}}{N^{-1}\sum_{i=1}^{N}x_{i}x_{i}}.$$ Here the numerator of $\hat{\beta}_{N}$ is your $r_{N}$. I did not impose any distribution assumptions about $x_{i}$ and $y_{i}$, so no loss of generality was incurred. Based on this linkage, we may approach your question by using the distribution of $\hat{\beta}_{N}$. However, it is well known that the finite sample distribution of $\hat{\beta}_{N}$ is not available in most cases. Consequently, my conclusion is pessimistic in that the 'exact' answer to your question might not exist in most cases. But an approximate $\tilde{N}$ can be easily found since $\hat{\beta}_{N}$ is asymptotically normal.
First, consider a finite sample case. If $\left\{ x_{i}\right\} _{i=1}^{N}$ is fixed in your experiment, and $\left\{ y_{i}\right\} _{i=1}^{N}$ are i.i.d. normal with mean $0$ and variance $\sigma^{2}$. Then it is well known that $$\hat{\beta}_{N}\mid x_{1},\ldots,x_{N}\sim N\left(\beta,\frac{\sigma^{2}}{N}\left(\frac{\sum_{i=1}^{N}x_{i}^{2}}{N}\right)^{-1}\right).$$ Consequently, $$r_{N}\mid x_{1},\ldots,x_{N}\sim N\left(0,\frac{\sigma^{2}}{N}\left(\frac{\sum_{i=1}^{N}x_{i}^{2}}{N}\right)\right).$$ Denote $\sigma_{r,N}^{2}=\left(\sigma^{2}/N\right)\left(\sum_{i=1}^{N}x_{i}^{2}/N\right)$, and note that $\sigma_{r,N}$ is monotone decreasing with respect to $N$. Then given $\varepsilon$, we have $\Pr\left(\left|r_{N}\right|<\varepsilon\right)=1-2\Phi\left(-\varepsilon/\sigma_{r,N}\right)$. By letting $$$1-2\Phi\left(-\varepsilon/\sigma_{r,N}\right)=\alpha, you can solve \tilde{N}. (Remark: I think you are asking for such a \tilde{N} that \Pr\left(\left|r_{N}\right|<\varepsilon\right)\geq\alpha for all N>\tilde{N}.) But these arguments are based on the properties of normal distributions. Next, consider the large sample case (N\rightarrow \infty), where x_i could be random. Suppose \left\{ x_{i}\right\} _{i=1}^{N} and \left\{ y_{i}\right\} _{i=1}^{N} satisfy the regularity conditions of the LLN and CLT. Asymptotically, we have$$ \hat{\beta}_{N}\rightarrow_{d}N\left(\beta,\frac{\sigma^{2}}{N}Q^{-1}\right),\quad Q=\mathrm{{plim}}_{N\rightarrow\infty}\frac{\sum_{i=1}^{N}x_{i}^{2}}{N}, $$where \sigma^{2} is the variance of y_{i}. By Slutsky lemma, we have$$ r_{N}\rightarrow_{d}N\left(0,\frac{\sigma^{2}}{N}Q\right).$$It is obvious that$\sigma_{r,N}^{2}$is a consistent estimator of$\left(\sigma^{2}/N\right)Q$. Then an approximate$\tilde{N}$can be found in the same way. • You seem to assume that the$x_i$are not random: I cannot find any place where you either justify or relax the assumption that "$\{x_i\}_{i=1}^N$is fixed in your experiment." Wouldn't it be simpler (and more accurate) just to compute the mean and variance of$Z = XY$and apply the CLT directly to it? Incidentally, closed-form analytical solutions for the distribution of$r_N$can be found when the$x_i$and$y_i$have Normal distributions ($|XY|$then has a Bessel(0) distribution). – whuber Commented Jul 17, 2013 at 21:00 • I did not assume$x_i$are nonrandom in the large sample case. But I did assume that$x_i$are nonrandom in the finite sample case. I should mention this. Let me edit the post a little bit. I am sure you would get the solution by applying the CLT directly to$XY\$. Commented Jul 18, 2013 at 0:06