7
$\begingroup$

I am computing the sample correlation between two vectors of uncorrelated and uniformly distributed samples using MATLAB. More precisely, I compute $$ r_N=\frac{1}{N}\sum_{i=1}^N x_{i}\, y_{i}, $$ where $E(X)=E(Y)={\rm cov}(X,Y)=0$, and $x_{i}, y_{i}$ are drawn from a uniform distribution in $[-A/2,A/2]$. It is clear that $\lim_{N \rightarrow \infty} r_N=0$. However, I would like to compute an estimate of $\tilde{N}$ such that, given $\epsilon >0$ and $\alpha >0$, we have $ {\rm Pr}(|r_N| < \epsilon) < \alpha$ for all $ N > \tilde{N}$.

In other words, I need an expression for the sample statistic $r_N$. I formulated this question for uniformly distributed data; however I'm interested to know what happens for other distributions (say normal).

Any ideas?

$\endgroup$
7
  • 2
    $\begingroup$ Since $x$ and $y$ have zero mean $r_N$ has the same expression of $r_{xy}$ in http://en.wikipedia.org/wiki/Correlation minus the normalization by the sample standard deviations so I'm not sure about why you don't see the correlation @COOLSerdash. $\endgroup$ Jul 12, 2013 at 22:52
  • 1
    $\begingroup$ You're looking at sampling error. Watch what happens when you increase $N$ to $10^6$, then $10^7$, then $10^8$, etc. $\endgroup$
    – whuber
    Jul 13, 2013 at 2:26
  • 2
    $\begingroup$ I see your point @whuber. So my question now is: is there a way to estimate $N$ such that $P(\left|r_N\right|<\epsilon)>p$ ? $\endgroup$ Jul 13, 2013 at 16:19
  • 2
    $\begingroup$ It's not clear that $\lim_{N\to\infty} r_N = 0$. Indeed, $r_N$ is an estimator of ${\rm E}(XY)$, and ${\rm E}(XY) = {\rm cov}(X, Y)$ since ${\rm E}(X)={\rm E}(X)=0$. So, if ${\rm cov}(X, Y) \neq 0$ then $\lim_{N\to\infty} r_N \neq 0$. This implies that $\tilde{N}$ might not exist. Is the covariance/correlation between $X$ and $Y$ supposed to be zero? $\endgroup$
    – QuantIbex
    Jul 16, 2013 at 23:30
  • 2
    $\begingroup$ Yes, I assume that ${\rm cov}(X,Y)=0$ so I added that to the question. $\endgroup$ Jul 17, 2013 at 16:19

1 Answer 1

2
$\begingroup$

Your question is indeed asking for the finite sample distribution of $r_{N}$. To address your question, let me rephrase it in terms of linear regressions. So a linkage between $r_{N}$ and the ordinary least square (OLS) estimator could be highlighted. You observe two variables $\left\{ x_{i}\right\} _{i=1}^{N}$ and $\left\{ y_{i}\right\} _{i=1}^{N}$, and they are uncorrelated. Suppose you run a linear regression between $y_{i}$ and $x_{i}$, $$ y_{i}=\beta x_{i}+u_{i},\quad i=1,\ldots,N. $$ Since $y_{i}$ and $x_{i}$ are uncorrelated, the true value $\beta$ is $0$. Note that the OLS estimator of $\beta$ is $$ \hat{\beta}_{N}=\frac{\sum_{i=1}^{N}x_{i}y_{i}}{\sum_{i=1}^{N}x_{i}x_{i}}=\frac{N^{-1}\sum_{i=1}^{N}x_{i}y_{i}}{N^{-1}\sum_{i=1}^{N}x_{i}x_{i}}. $$ Here the numerator of $\hat{\beta}_{N}$ is your $r_{N}$. I did not impose any distribution assumptions about $x_{i}$ and $y_{i}$, so no loss of generality was incurred. Based on this linkage, we may approach your question by using the distribution of $\hat{\beta}_{N}$. However, it is well known that the finite sample distribution of $\hat{\beta}_{N}$ is not available in most cases. Consequently, my conclusion is pessimistic in that the 'exact' answer to your question might not exist in most cases. But an approximate $\tilde{N}$ can be easily found since $\hat{\beta}_{N}$ is asymptotically normal.

First, consider a finite sample case. If $\left\{ x_{i}\right\} _{i=1}^{N}$ is fixed in your experiment, and $\left\{ y_{i}\right\} _{i=1}^{N}$ are i.i.d. normal with mean $0$ and variance $\sigma^{2}$. Then it is well known that $$ \hat{\beta}_{N}\mid x_{1},\ldots,x_{N}\sim N\left(\beta,\frac{\sigma^{2}}{N}\left(\frac{\sum_{i=1}^{N}x_{i}^{2}}{N}\right)^{-1}\right). $$ Consequently, $$ r_{N}\mid x_{1},\ldots,x_{N}\sim N\left(0,\frac{\sigma^{2}}{N}\left(\frac{\sum_{i=1}^{N}x_{i}^{2}}{N}\right)\right). $$ Denote $\sigma_{r,N}^{2}=\left(\sigma^{2}/N\right)\left(\sum_{i=1}^{N}x_{i}^{2}/N\right)$, and note that $\sigma_{r,N}$ is monotone decreasing with respect to $N$. Then given $\varepsilon$, we have $\Pr\left(\left|r_{N}\right|<\varepsilon\right)=1-2\Phi\left(-\varepsilon/\sigma_{r,N}\right)$. By letting $ $$1-2\Phi\left(-\varepsilon/\sigma_{r,N}\right)=\alpha$, you can solve $\tilde{N}$. (Remark: I think you are asking for such a $\tilde{N}$ that $\Pr\left(\left|r_{N}\right|<\varepsilon\right)\geq\alpha$ for all $N>\tilde{N}$.) But these arguments are based on the properties of normal distributions.

Next, consider the large sample case ($N\rightarrow \infty$), where $x_i$ could be random. Suppose $\left\{ x_{i}\right\} _{i=1}^{N}$ and $\left\{ y_{i}\right\} _{i=1}^{N}$ satisfy the regularity conditions of the LLN and CLT. Asymptotically, we have $$ \hat{\beta}_{N}\rightarrow_{d}N\left(\beta,\frac{\sigma^{2}}{N}Q^{-1}\right),\quad Q=\mathrm{{plim}}_{N\rightarrow\infty}\frac{\sum_{i=1}^{N}x_{i}^{2}}{N}, $$ where $\sigma^{2}$ is the variance of $y_{i}$. By Slutsky lemma, we have $$ r_{N}\rightarrow_{d}N\left(0,\frac{\sigma^{2}}{N}Q\right). $$ It is obvious that $\sigma_{r,N}^{2}$ is a consistent estimator of $\left(\sigma^{2}/N\right)Q$. Then an approximate $\tilde{N}$ can be found in the same way.

$\endgroup$
2
  • 2
    $\begingroup$ You seem to assume that the $x_i$ are not random: I cannot find any place where you either justify or relax the assumption that "$\{x_i\}_{i=1}^N$ is fixed in your experiment." Wouldn't it be simpler (and more accurate) just to compute the mean and variance of $Z = XY$ and apply the CLT directly to it? Incidentally, closed-form analytical solutions for the distribution of $r_N$ can be found when the $x_i$ and $y_i$ have Normal distributions ($|XY|$ then has a Bessel(0) distribution). $\endgroup$
    – whuber
    Jul 17, 2013 at 21:00
  • 1
    $\begingroup$ I did not assume $x_i$ are nonrandom in the large sample case. But I did assume that $x_i$ are nonrandom in the finite sample case. I should mention this. Let me edit the post a little bit. I am sure you would get the solution by applying the CLT directly to $XY$. $\endgroup$
    – semibruin
    Jul 18, 2013 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.