Suppose we have a standard regression model
$$Y= X\beta + \epsilon$$
with $$\epsilon \sim \sigma^2$$ $$X \sim N(\mu,\gamma^2)$$
Are the estimated coefficients the same as if $X$ was fixed?
Is the estimated variance of $\hat{\beta}$ the same?
Thanks!
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
If we assume that $$ {\rm E}[\varepsilon | X] = 0, $$ then the regression coefficients are still calculated as $$ \hat{\beta} = (X^TX)^{-1}X^TY. $$ The variance of $\hat{\beta}$, however, is no longer the same as the one in the deterministic case since expression $$ \sigma^2(X^TX)^{-1} =: {\rm Var}[\hat{\beta} | X] $$ is now a random variable. To calculate the overall, unconditional variance of $\hat{\beta}$, we use the rule $$ {\rm Var}[\hat{\beta}] = {\rm E}[{\rm Var}[\hat{\beta} | X]] + {\rm Var}[{\rm E}[\hat{\beta} | X]]. $$ The landmark book by
... Greene, W. H. (2011). Econometric Analysis (7th ed). Upper Saddle River, NJ: Prentice Hall.
covers the case of stochastic predictors in detail in the earlier chapters.
$\begingroup$
$\endgroup$
Regression is the analysis of the conditional distribution of a response variable $Y$ given an explanatory variable $X$. The parameters in the regression model are measuring terms in the expectation of the response variable conditional on the explanatory variable, so the model, and the corresponding estimation method, so it will always treat the explanatory variable as a constant.
To the extent that you are interested in the marginal distribution of $Y$ when $X$ is a random variable, that is no longer a question about regression analysis --- it is multivariate analysis. In that case you formulate a broader model that will its own parameters to describe the marginal distribution of $Y$.
