8
$\begingroup$

We have a simple linear regression model. Our assumptions are:

$Y_i =\beta_0+\beta_1X_i+ \varepsilon_i $, $i=1, \cdots, n$

$\varepsilon_i \sim N(0, \sigma^2)$

$Var(\varepsilon_i|X_i=x)=\sigma^2$

$\varepsilon_1, \cdots, \varepsilon_n$ are mutually independent.

$\\$

Are these hypothesis enough to claim that $\varepsilon_i|X_i=x \sim N(0, \sigma^2)$?

$\endgroup$
10
  • $\begingroup$ In $\operatorname{Var}(\varepsilon_i|X=x)=\sigma^2$, $X$ does not have a subscript. Is that intentional? Also, in the very last formula $X$ does not have a subscript. Again, is that intentional? Are $X$ and $x$ vectors in both cases? $\endgroup$ – Richard Hardy Sep 9 '15 at 17:53
  • $\begingroup$ @RichardHardy They are note vectors, the OP says "simple" linear regression, meaning only one explanatory variable, besides the constant term. $\endgroup$ – Alecos Papadopoulos Sep 9 '15 at 18:03
  • $\begingroup$ @RichardHardy No, it was a mistake. $\endgroup$ – DGRasines Sep 9 '15 at 18:29
  • $\begingroup$ @AlecosPapadopoulos, I did notice that was a simple regression but then $X$ means a column vector and $X_i$ means one element of a vector. As the author noted, writing $X$ rather than $X_i$ was a mistake. $\endgroup$ – Richard Hardy Sep 9 '15 at 18:34
  • 1
    $\begingroup$ Your title and your body questions don't seem to be asking the same thing. $\endgroup$ – Glen_b Sep 9 '15 at 23:28
12
$\begingroup$

No. Here's an interesting counterexample.

Define a distribution function

$$g(x) = \frac{2}{\sqrt{2\pi}}\exp(-x^2/2)I(-t \le x \le 0 \text{ or } t \le x)$$

for $t = \sqrt{2\log(2)} \approx 1.17741$. ($I$ is the indicator function.)

The plot of $g$ is shown here in blue. If we define $h(x) = g(-x)$, its plot appears in red.

Figure

Direct calculation shows that any variable $Y$ with distribution $g$ has zero mean and unit variance. By construction, an equal mixture of $Y$ with $-Y$ (whose PDF is $h$) has a distribution function proportional to $\exp(-x^2/2)$: that is, it is standard Normal (with zero mean and unit variance).

Let $X_i$ have a Bernoulli$(1/2)$ distribution. Suppose $\varepsilon_i|X=0$ has distribution $g$ and $\varepsilon_i|X=1$ has distribution $h$, with all the $(X_i, \varepsilon_i)$ independent. The assumption about $Y_i$ is irrelevant (or true by definition of $Y_i$) and all the other assumptions hold by construction, yet none of the conditional distributions $\varepsilon_i | X_i = x$ are Normal for any value of $x$.

$\endgroup$
2
  • $\begingroup$ (+1) This is the true value-added of CV. Thanks for this. $\endgroup$ – Alecos Papadopoulos Sep 10 '15 at 1:20
  • $\begingroup$ The fun in math... So good! $\endgroup$ – Antoni Parellada Sep 10 '15 at 1:32
4
$\begingroup$

The assumption that the conditional variance is equal to the unconditional variance, together with the assumption that $E(\varepsilon_i)=0$, does imply zero conditional mean, namely

$$\{{\rm Var}(\varepsilon_i \mid X_i) = {\rm Var}(\varepsilon_i)\} \;\text {and}\;\{E(\varepsilon_i)=0\}\implies E(\varepsilon_i \mid X_i)=0 \tag{1}$$

The two assumptions imply that

$$E(\varepsilon_i^2 \mid X_i) -[E(\varepsilon_i \mid X_i]^2 = E(\varepsilon_i^2)$$ $$\implies E(\varepsilon_i^2 \mid X_i) - E(\varepsilon_i^2) = [E(\varepsilon_i \mid X_i]^2$$

Ad absurdum, assume that $E(\varepsilon_i \mid X_i)\neq 0 \implies [E(\varepsilon_i \mid X_i]^2 >0$

This in turn implies that $E(\varepsilon_i^2 \mid X_i) > E(\varepsilon_i^2)$. By the law of iterated expectations we have $E(\varepsilon_i^2) = E\big[ E(\varepsilon_i^2 \mid X_i)\big]$. For clarity set $Z \equiv E(\varepsilon_i^2 \mid X_i)$. Then we have that

$$E(\varepsilon_i \mid X_i)\neq 0 \implies Z > E(Z)$$

But this cannot be since a random variable cannot be strictly greater than its own expected value. So $(1)$ must hold.

Note that the reverse is not necessarily true.

As for providing an example to show that even if the above results hold, and even under the marginal normality assumption, the conditional distribution is not necessarily identical to the marginal (which would establish independence), whuber beat me to it.

$\endgroup$
12
  • 1
    $\begingroup$ Let us add an additional assumption that the true model is indeed linear and $\operatorname{E}(\varepsilon_i|X_i)=0$. We then have 1. unconditional normality, 2. conditional mean zero and 3. conditional constant variance. I still do not see whether (and if so, how) that yields conditional normality. (The sequence would work fine the other way around: conditional normality + conditional mean zero + conditional constant variance --> {unconditional normality+mean zero+constant variance.}) $\endgroup$ – Richard Hardy Sep 9 '15 at 18:37
  • $\begingroup$ @RichardHardy Where in my answer do I write that conditional normality follows from the assumptions that you state? I explicitly write "conditional normality follows from nowhere". $\endgroup$ – Alecos Papadopoulos Sep 9 '15 at 18:49
  • $\begingroup$ True. But you "blame" the fact that the true model may be nonlinear. If I may add this extra assumption about the true model and reiterate the question, would your answer change? $\endgroup$ – Richard Hardy Sep 9 '15 at 18:54
  • $\begingroup$ @RichardHardy Where do I do that? The linearity/non-linerariy has to do with the mean-independence, not with conditional normality. $\endgroup$ – Alecos Papadopoulos Sep 9 '15 at 18:56
  • 1
    $\begingroup$ Thanks, I thought so, too. However, I was unable to come up with a simple counterexample where all the conditions (plus the one stating that the true model is linear) are satisfied but the conditional distribution is nonnormal. A counterexample is always a nice way to disprove a hypothesis, so I thought it would be nice to come up with one. Anyway, yours is a nice answer. $\endgroup$ – Richard Hardy Sep 9 '15 at 20:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.