1
$\begingroup$

I was reading about white noise and it stated:

Although $\varepsilon_t$ & $y_t$ are serially uncorrelated, they are not necessarily serially independent, because they are not necessarily normally distributed. If in addition to being serially uncorrelated, $y$ is serially independent, then we say $y$ is independent white noise.

I cannot understand the link between the normal distribution and being serially independent. Can anyone help me with this?

$\endgroup$
  • $\begingroup$ @Dilip The statement is correct, because it does not assert (or even imply) that uncorrelated normal variables are independent. It only relies on the fact that it is invalid to infer independence from lack of correlation in non-normal variates. Arka, your question is addressed in many threads here. The one at stats.stackexchange.com/questions/4364 appears to contain some explicit answers. $\endgroup$ – whuber Aug 8 '15 at 15:32
  • $\begingroup$ @whuber I don't think any of the answers to the question you referred to ... q/4364/6633 or comments thereon have any relevance to Arka's question. Could you be more specific as to which answer or comment you are saying is an explicit answer to Arka's question? $\endgroup$ – Dilip Sarwate Aug 9 '15 at 16:10
  • $\begingroup$ @Dilip stats.stackexchange.com/a/4837 concerns the relationship between correlation and independence in Normal variables. $\endgroup$ – whuber Aug 9 '15 at 18:58
  • $\begingroup$ @whuber I am still not understanding how ""Let $X_1, \ldots, X_n$ be independent random variables. Then $\sum_{i=1}^n{a_i x_i}$ and $\sum_{i=1}^n{b_i x_i}$ are independent, where $a_i b_i \ne 0$, if and only if $X_i$ [are] normally distributed." is an answer to the question being asked here (or indeed is even a true statement since with $a_i=b_i\neq 0$, it seems to be saying that $\sum_{i=1}^n a_iX_i$ is independent of itself exactly when all the $X_i$ are normal). However, I bow to your superior knowledge and wisdom and accept your contention that it is exactly what the OP needs to know. $\endgroup$ – Dilip Sarwate Aug 10 '15 at 3:04
  • 1
    $\begingroup$ @Dilip This is not an issue that could be settled by any amount of "superior knowledge" or "wisdom," because I think we may be discussing different interpretations of the question. $\endgroup$ – whuber Aug 10 '15 at 13:22
2
$\begingroup$

My interpretation of the (slightly paraphrased) statement that the OP is reading, viz.

"If $\{Y_n\}$ is a sequence of (serially) uncorrelated random variables, then $\{Y_n\}$ is not necessarily a sequence of independent random variables, because (emphasis added) they are not necessarily normally distributed"

is that the author is asserting (quite correctly) that uncorrelated random variables need not be independent but the reason for this failure of uncorrelatedness to imply independence is that we cannot assert that the uncorrelated random variables are normally distributed. If the author is not implying that uncorrelated random variables that are normally distributed are independent random variables, then he is certainly begging the reader to jump to that (false) conclusion, by musing in the the subjunctive mood: "Gol dang it, if only those pesky $Y_n$'s were normally distributed in addition to being uncorrelated, then we could take those uncorrelated (and normal) $Y_n$'s to be independent random variables and avoid a lot of headaches." However, Moderator @whuber has stated (in a comment on the main question) that he does not interpret that sentence that way, and that the statement quoted by the OP is perfectly accurate.

In my opinion, the second sentence quoted by the OP.

If in addition to being serially uncorrelated, the $\{Y_n\}$ are serially independent, then we say $\{Y_n\}$ is independent white noise.

also incorrect. Independent random variables are always uncorrelated and it is unnecessary to start with uncorrelated random variables and then impose the additional constraint that they are independent random variables. Furthermore, if by serially independent it is meant that for all $n\neq m$, $Y_m$ and $Y_n$ are independent random variables (that is, only pairwise independence is required), then I disagree vehemently with the assertion that $\{Y_n\}$ is a white noise process. For a random process to be called a white noise process, the random variables need to be mutually independent, not just pairwise independent, and most people, upon encountering the phrase white notes process, are likely to assume that the the random variables constituting the white noise process also are zero-mean random variables with common finite variance $\sigma^2$. This property of the $Y_n$'s is nowhere mentioned in the paragraph fragment quoted by the OP.


Finally, turning to the OP's complaint

I cannot understand the link between the normal distribution and being serially independent

I say that it is a red herring. Uncorrelated (marginally) normal random variables are not necessarily independent random variables while uncorrelated jointly normal random random variables are always independent random variables.

It is not true that if $\{Y_n\}$ is a sequence of normally distributed random variables that happen to be uncorrelated, then the random variables are independent.

A standard counterexample begins with $X \sim N(0,1)$ and an independent random variable $Z$ that takes on values $+1$ and $-1$ with equal probability. Then $Y = XZ$ is also a standard normal random variable since \begin{align}P\{Y \leq x\} &= P\{X \leq x\mid Z=+1\}P\{Z=+1\} + P\{X \geq -x\mid Z=-1\}P\{Z=-1\}\\ &= \frac 12 \Phi(x) + \frac 12 (1 - \Phi(-x))\\ &= \Phi(x). \end{align} Also, $\quad\operatorname{cov}(X,Y) = E[XY]-E[X]E[Y]= E[X^2Z]=E[X^2]E[Z]=0$ showing that $X$ and $Y$ are are uncorrelated random variables. But $X$ and $Y$, although they are uncorrelated normal random variables, are not independent random variables but instead very much dependent random variables since given that $X=x$, $Y$ takes on values $\pm x$ with equal probability $\frac 12$.

Now, with $\{Z_n\}$ being a sequence of independent random variables with the same distribution as $Z$ (and all independent of $X$ also), set $Y_n = XZ_n$. It follows from our construction that $Y_n \sim N(0,1)$. But, $$E[Y_nY_{m}] = E[X^2Z_nZ_m] = E[X^2]E[Z_n]E[Z_m] = 0~\text{provided that} ~ n \neq m.$$ Thus, the $\{Y_n\}$ are uncorrelated and normal. But they are not independent because if we know that $Y_m = y$, then we know that $Y_n$ is necessarily either $y$ of $-y$.

Normal random variables need not be jointly normal random variables and it is only in the case of joint normality that one can assert that uncorrelated (jointly) normal random variables are independent.

$\endgroup$
  • 1
    $\begingroup$ +1 Although I did read the question differently, you have done a great job of explaining--and justifying--how you read it and of raising (and answering) the attendant questions. I am sorry I overlooked your answer until now. $\endgroup$ – whuber May 12 '16 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.