0
$\begingroup$

I intended to derive the large sample distribution of the intercept of the least squares estimator $\beta_0$, but after many tries I could not obtain the solution that is given in the book of Stock&Watson as is depicted below.

enter image description here

I have been able to derive the large sample distribution of the slope $\beta_1$ myself, so we could treat that as a known random variable.

$\hat{\beta}_1 \sim N(\beta_1,\sigma_{\hat{\beta_1}}^{2})$

$\hat{\beta}_0$ have I earlier defined as $\hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}$

The approach I chose was to treat $\hat{\beta}_0$ as a function of $\hat{\beta}_1$

We know that $\bar{Y}$ and $\bar{X}$ are consistent estimators of their true mean $\mu_X$ and $\mu_Y$ by means of LLN, so the equation becomes the following (given n is large). The assumptions of LLN are assumed to be all statisfied.

$\hat{\beta}_0 = \mu_Y - \hat{\beta}_1 \mu_X$

$\hat{\beta}_0 \sim N(\mu_Y - \beta_1 \mu_X,\frac{\sigma_{\hat{\beta_1}}^{2}}{(n\sigma_x^2)^2})$

Could anyone shed some light to this issue I am facing. I hope someone can help me out!

EDIT

$\bar{X}\sim N(\mu_X,\sigma_x^2/n)$ (assuming the case N is large and $X_i$ is i.i.d., $\bar{X}$ is a consistent estimator, and approximately normally distributed due to CLT)

$\bar{Y}\sim N(\mu_Y,\sigma_y^2/n)$

$\sigma_{\hat{\beta_1}}^2=\frac{1}{n}\frac{var((X_i-\mu_x)u_i)}{(var(x_i))^2}$

$u_i=Y_i-\beta_0-\beta_1 X_i$ (the unobserved error term of the simple linear regression model)

EDIT 2:

$\hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}$

$var(\hat{\beta}_0) = var(\bar{Y} - \hat{\beta}_1 \bar{X})$

$ var(\hat{\beta}_0) = var(Y)/n + \bar{X}^2 var(\hat{\beta}_1)$

Note that X is regarded as nonrandom: $var(Y) = var( \beta_0 + \beta_1X + u) = var(u)$

$var(\hat{\beta}_0) = \frac{var(u)}{n} + \bar{X}^2 var(\hat{\beta}_1)$

$var(\hat{\beta}_0) = \frac{var(u)}{n} + \frac{\bar{X}^2}{n} \frac{var((X-\mu_x)u)}{var(x)^2}$

$var(\hat{\beta}_0) = \frac{1}{n} \Big( var(u)+ \bar{X}^2 \frac{var((X-\mu_x)u)}{var(x)^2} \Big)$

$var(\hat{\beta}_0) = \frac{1}{n} \Big( var(u)+ \bar{X}^2 \frac{var(X)var(u)+\mu_x^2var(u)}{var(x)^2} \Big)$

$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ \bar{X}^2 \frac{var(X)+\mu_x^2}{var(x)^2} \Big)$

Then one might to say that $\bar{X}$ is a consistent estimator of $\mu_x$ given that n is large and $X$ is iid.

$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ \mu_x^2 \frac{var(X)+\mu_x^2}{var(x)^2} \Big)$

$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ E(X)^2 \frac{E(X^2)}{var(x)^2} \Big)$

$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ \frac{E(X)^2E(X^2)}{(E(X^2)-E(X))^2} \Big)$

The final expression looks already very familiar with the $H_i$ in the textbook. But I am not there yet.

$\endgroup$

1 Answer 1

0
$\begingroup$

These are not the usual assumptions for linear regression. $\bar{X}$ is not usually assumed to have a normal distribution. The $X_i$ are assumed fixed. $Var[Y_i]$ is assumed to be $\sigma^2$. I don't know what $u_i$ means. Also, I don't know what it means to use subscripts $i$ in the formula for the variances without a summation symbol.

$$E[\hat{\beta_0}-\beta_0]=E[\bar{Y}-\hat{\beta_1}\bar{X}-\beta_0]=E[\bar{Y}]-\beta_1\bar{X}-\beta_0$$
$$=\frac{1}{n}\sum_{i=1}^n{\left(\beta_1 x_i+\beta_0\right)}-\beta_1\bar{X}-\beta_0=0$$

To find $Var[\hat{\beta_0}]$, I don't think you don't have enough information from what is shown there to calculate it.

The easiest way to derive the variance of both $\beta_0$ and $\beta_1$ and the correlation between them is shown here.

I can't find the variance from the information provided there. But, if I assume the formula for $\hat{\beta_1}$ is known, then I can find it this way. All sums are from $i=1,...,n$.

$$Var[\hat{\beta_0}]=Var[\bar{Y}-\hat{\beta_1}\bar{X}]$$
$$=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]-2 \bar{X} Cov[\bar{Y},\hat{\beta_1}]$$
$$=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]-2 \bar{X} Cov[\frac{1}{n} \sum{Y_i},\frac{\sum(X_i-\bar{X})(Y_i-\bar{Y})}{\sum{(X_i-\bar{X})^2}}]$$
$$=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]-\frac{2 \bar{X}}{n \sum{(X_i-\bar{X})^2}} Cov[ \sum{Y_i},\sum(X_i-\bar{X})(Y_i-\bar{Y})]$$

Now, since $\sum(X_i-\bar{X})]=0$, we have
$$Cov[ \sum{Y_i},\sum(X_i-\bar{X})(Y_i-\bar{Y})]$$ $$=Cov[ \sum{Y_i},\sum Y_i(X_i-\bar{X})-\bar{Y} \sum(X_i-\bar{X})]$$ $$=Cov[ \sum{Y_i},\sum Y_i(X_i-\bar{X})]$$ $$=\sum (X_i-\bar{X}) Var[Y_i]=\sum (X_i-\bar{X}) ] \sigma^2=0$$

So, all together,
$$Var[\hat{\beta_0}]=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]=\frac{\sigma^2}{n}+(\bar{X})^2 Var[\hat{\beta_1}]$$

From here, again, I do not see how to simplify this from what you are given. But, if you know that $Var[\hat{\beta_1}]=\frac{\sigma^2}{\sum(X_i-\bar{X})^2}$, then you can plug that in here and simplify it to get $Var[\hat{\beta_0}]=\frac{\sigma^2 \sum {X_i}^2}{n \sum(X_i-\bar{X})^2}=\frac{\sum {X_i}^2}{n } Var[\hat{\beta_1}]$.

$\endgroup$
6
  • $\begingroup$ Thanks for attempting to answer the question. I intentionally wanted to derive the variance of $\beta_0$ in scalar format, not in matrices. As of the potential information that is missing, I doubt that is the case. Some identities that I could further think of are given in the edit. $\endgroup$ Feb 11, 2021 at 17:59
  • $\begingroup$ I believe you made a mistake, the variance of a constant should be squared. The $\bar{X}$ should be $\bar{X}^2$. I managed to make a few others transformations in order to find the expression in the textbook for the variance of $\hat{\beta_0}$. However, still failed to obtain the final expression. $\endgroup$ Feb 15, 2021 at 9:59
  • $\begingroup$ @NadiaMerquez fixed now $\endgroup$
    – John L
    Feb 15, 2021 at 15:35
  • $\begingroup$ Thanks. Any idea how $H_i$ may be constructed based on the latest edit? $\endgroup$ Feb 15, 2021 at 20:22
  • $\begingroup$ @NadiaMerquez I don't know what the formula means because I don't understand how there can be a subscript $i$. Since there are many different values of $X_i$, there would have to be many different values of $H_i$. Then, I would get a different value of the variance depending on which one I picked. Unless it means the variance will be the same regardless of which $i$ I pick. But, that doesn't seem possible to me. $\endgroup$
    – John L
    Feb 15, 2021 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.