2
$\begingroup$

I refer here to a classical linear regression whose true representation is given by the equation: $y_i=x_i'\beta+u_i$,

where as usual $x_i$ is a $K\times 1$ vector of independent explanatory variables, $\beta$ is a $K\times 1$ vector of parameters and $u_i$ the error term of the ith observation and by construction of the true representation $u_i$ is considered to be independent of $x_i$.

My first question on that is basic: If we assume independence between $x_i$ and $u_i$ does this imply $E[u_i^d|x_i]=E[u_i^d]$ for $d=1,2,..$ - so to speak, can we generalize the Law of iterated expectations consequently to all moments?

Secondly, what is the basis of many textbooks to claim that $x_i$ and $u_i$ are independent in the classical regression model? By independence, I don't mean uncorrelatedness, rather $f(u,x)=f(u)f(x)$. I am aware that the two key standard assumptions on which this small sample properties model is based on is strict exogenity $E[u_i|X]=0$, and normally distributed error terms $u_i|x_i \thicksim N(0, \sigma^2)$. Where does the claim of the independence between $x_i$ and $u_i$ originates from in textbooks? Is it implied by these two statements jointly together or is this a separate assumption? If the latter, why is it not mentioned exclusively in the set of assumptions for classical linear regression models which only assumes strict exogenity and normally distributed errors?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Could you please cite some textbooks, which do in fact claim that $x_i$ and $u_i$ must be independent? Usually either $x_i$ are fixed, which means that they are independent of $u_i$, or $x_i$ are stochastic and then you need only that $Ex_iu_i=0$, independence condition is superfluous. $\endgroup$
    – mpiktas
    Commented Dec 18, 2013 at 12:03

1 Answer 1

Reset to default
1
$\begingroup$

The answer to the first part follows from the law of the iterated expectation for the mean and law of iterated variance for the variance. See here and here for the proofs [I am not sure about the third or higher moments]. For the second part, strict exogenity $E[u_i|X]=0$ implies $E[u_i,X]=0$, but the reverse is not true. To see why this is, the former also implies non-linear (or any function of $X$) correlation, e.g., $E[u_i,X^2]=0$.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Firstly, thanks for your answer! However, I am not sure if all of my questions are correctly addressed. I believe the Law of Iterated Expectations is for the first moment only. My question refers to the idea if we can generalize this law for higher moments given that we have independence between $u_i$ and $x_i$. The answer to my second question is perhaps not correct? I am asking if exogenity + normal distributed errors imply independence or otherwise. I am not sure what $E[u_i,X]$ is, do you mean $E[u_i X]$? If so, this would be the covariance moment, but this is not independence per se. $\endgroup$
    – Majte
    Commented Mar 22, 2013 at 21:42
  • 1
    $\begingroup$ There are cases in which uncorrelatedness does imply independence. See here. To quote "...two jointly normally distributed random variables are independent if they are uncorrelated,although this does not hold for variables whose marginal distributions are normal and uncorrelated but whose joint distribution is not joint normal)" $\endgroup$
    – Metrics
    Commented Mar 22, 2013 at 21:55
  • $\begingroup$ Which is my question from above: Does the classical linear regression model imply through exogenity uncorrelatedeness and by the normal error assumption true independence which in return implies higher moment unconditional AND conditional independence, i.e. $E[u_i^d|x_i]=E[u_i^d]$? If yes, I need a formal proof. But we are getting there :) $\endgroup$
    – Majte
    Commented Mar 22, 2013 at 22:04
  • $\begingroup$ Secondly, what is the basis of those two variables to be jointly normal? The classical linear regression model assumes X to be fixed, so there would be no joint distribution? $\endgroup$
    – Majte
    Commented Mar 22, 2013 at 22:09
  • 1
    $\begingroup$ X does not need to be fixed in the classical linear regression. That's what the exogenity is for, after all. $\endgroup$
    – IMA
    Commented May 22, 2013 at 11:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.