# Derivation of Large sample distribution for the least squares estimator of the intercept $\beta_0$

I intended to derive the large sample distribution of the intercept of the least squares estimator $$\beta_0$$, but after many tries I could not obtain the solution that is given in the book of Stock&Watson as is depicted below.

I have been able to derive the large sample distribution of the slope $$\beta_1$$ myself, so we could treat that as a known random variable.

$$\hat{\beta}_1 \sim N(\beta_1,\sigma_{\hat{\beta_1}}^{2})$$

$$\hat{\beta}_0$$ have I earlier defined as $$\hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}$$

The approach I chose was to treat $$\hat{\beta}_0$$ as a function of $$\hat{\beta}_1$$

We know that $$\bar{Y}$$ and $$\bar{X}$$ are consistent estimators of their true mean $$\mu_X$$ and $$\mu_Y$$ by means of LLN, so the equation becomes the following (given n is large). The assumptions of LLN are assumed to be all statisfied.

$$\hat{\beta}_0 = \mu_Y - \hat{\beta}_1 \mu_X$$

$$\hat{\beta}_0 \sim N(\mu_Y - \beta_1 \mu_X,\frac{\sigma_{\hat{\beta_1}}^{2}}{(n\sigma_x^2)^2})$$

Could anyone shed some light to this issue I am facing. I hope someone can help me out!

EDIT

$$\bar{X}\sim N(\mu_X,\sigma_x^2/n)$$ (assuming the case N is large and $$X_i$$ is i.i.d., $$\bar{X}$$ is a consistent estimator, and approximately normally distributed due to CLT)

$$\bar{Y}\sim N(\mu_Y,\sigma_y^2/n)$$

$$\sigma_{\hat{\beta_1}}^2=\frac{1}{n}\frac{var((X_i-\mu_x)u_i)}{(var(x_i))^2}$$

$$u_i=Y_i-\beta_0-\beta_1 X_i$$ (the unobserved error term of the simple linear regression model)

EDIT 2:

$$\hat{\beta}_0 = \bar{Y} - \hat{\beta}_1 \bar{X}$$

$$var(\hat{\beta}_0) = var(\bar{Y} - \hat{\beta}_1 \bar{X})$$

$$var(\hat{\beta}_0) = var(Y)/n + \bar{X}^2 var(\hat{\beta}_1)$$

Note that X is regarded as nonrandom: $$var(Y) = var( \beta_0 + \beta_1X + u) = var(u)$$

$$var(\hat{\beta}_0) = \frac{var(u)}{n} + \bar{X}^2 var(\hat{\beta}_1)$$

$$var(\hat{\beta}_0) = \frac{var(u)}{n} + \frac{\bar{X}^2}{n} \frac{var((X-\mu_x)u)}{var(x)^2}$$

$$var(\hat{\beta}_0) = \frac{1}{n} \Big( var(u)+ \bar{X}^2 \frac{var((X-\mu_x)u)}{var(x)^2} \Big)$$

$$var(\hat{\beta}_0) = \frac{1}{n} \Big( var(u)+ \bar{X}^2 \frac{var(X)var(u)+\mu_x^2var(u)}{var(x)^2} \Big)$$

$$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ \bar{X}^2 \frac{var(X)+\mu_x^2}{var(x)^2} \Big)$$

Then one might to say that $$\bar{X}$$ is a consistent estimator of $$\mu_x$$ given that n is large and $$X$$ is iid.

$$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ \mu_x^2 \frac{var(X)+\mu_x^2}{var(x)^2} \Big)$$

$$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ E(X)^2 \frac{E(X^2)}{var(x)^2} \Big)$$

$$var(\hat{\beta}_0) = \frac{1}{n} var(u) \Big( 1+ \frac{E(X)^2E(X^2)}{(E(X^2)-E(X))^2} \Big)$$

The final expression looks already very familiar with the $$H_i$$ in the textbook. But I am not there yet.

These are not the usual assumptions for linear regression. $$\bar{X}$$ is not usually assumed to have a normal distribution. The $$X_i$$ are assumed fixed. $$Var[Y_i]$$ is assumed to be $$\sigma^2$$. I don't know what $$u_i$$ means. Also, I don't know what it means to use subscripts $$i$$ in the formula for the variances without a summation symbol.

$$E[\hat{\beta_0}-\beta_0]=E[\bar{Y}-\hat{\beta_1}\bar{X}-\beta_0]=E[\bar{Y}]-\beta_1\bar{X}-\beta_0$$
$$=\frac{1}{n}\sum_{i=1}^n{\left(\beta_1 x_i+\beta_0\right)}-\beta_1\bar{X}-\beta_0=0$$

To find $$Var[\hat{\beta_0}]$$, I don't think you don't have enough information from what is shown there to calculate it.

The easiest way to derive the variance of both $$\beta_0$$ and $$\beta_1$$ and the correlation between them is shown here.

I can't find the variance from the information provided there. But, if I assume the formula for $$\hat{\beta_1}$$ is known, then I can find it this way. All sums are from $$i=1,...,n$$.

$$Var[\hat{\beta_0}]=Var[\bar{Y}-\hat{\beta_1}\bar{X}]$$
$$=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]-2 \bar{X} Cov[\bar{Y},\hat{\beta_1}]$$
$$=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]-2 \bar{X} Cov[\frac{1}{n} \sum{Y_i},\frac{\sum(X_i-\bar{X})(Y_i-\bar{Y})}{\sum{(X_i-\bar{X})^2}}]$$
$$=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]-\frac{2 \bar{X}}{n \sum{(X_i-\bar{X})^2}} Cov[ \sum{Y_i},\sum(X_i-\bar{X})(Y_i-\bar{Y})]$$

Now, since $$\sum(X_i-\bar{X})]=0$$, we have
$$Cov[ \sum{Y_i},\sum(X_i-\bar{X})(Y_i-\bar{Y})]$$ $$=Cov[ \sum{Y_i},\sum Y_i(X_i-\bar{X})-\bar{Y} \sum(X_i-\bar{X})]$$ $$=Cov[ \sum{Y_i},\sum Y_i(X_i-\bar{X})]$$ $$=\sum (X_i-\bar{X}) Var[Y_i]=\sum (X_i-\bar{X}) ] \sigma^2=0$$

So, all together,
$$Var[\hat{\beta_0}]=Var[\bar{Y}]+(\bar{X})^2 Var[\hat{\beta_1}]=\frac{\sigma^2}{n}+(\bar{X})^2 Var[\hat{\beta_1}]$$

From here, again, I do not see how to simplify this from what you are given. But, if you know that $$Var[\hat{\beta_1}]=\frac{\sigma^2}{\sum(X_i-\bar{X})^2}$$, then you can plug that in here and simplify it to get $$Var[\hat{\beta_0}]=\frac{\sigma^2 \sum {X_i}^2}{n \sum(X_i-\bar{X})^2}=\frac{\sum {X_i}^2}{n } Var[\hat{\beta_1}]$$.

• Thanks for attempting to answer the question. I intentionally wanted to derive the variance of $\beta_0$ in scalar format, not in matrices. As of the potential information that is missing, I doubt that is the case. Some identities that I could further think of are given in the edit. Commented Feb 11, 2021 at 17:59
• I believe you made a mistake, the variance of a constant should be squared. The $\bar{X}$ should be $\bar{X}^2$. I managed to make a few others transformations in order to find the expression in the textbook for the variance of $\hat{\beta_0}$. However, still failed to obtain the final expression. Commented Feb 15, 2021 at 9:59
• @NadiaMerquez fixed now Commented Feb 15, 2021 at 15:35
• Thanks. Any idea how $H_i$ may be constructed based on the latest edit? Commented Feb 15, 2021 at 20:22
• @NadiaMerquez I don't know what the formula means because I don't understand how there can be a subscript $i$. Since there are many different values of $X_i$, there would have to be many different values of $H_i$. Then, I would get a different value of the variance depending on which one I picked. Unless it means the variance will be the same regardless of which $i$ I pick. But, that doesn't seem possible to me. Commented Feb 15, 2021 at 22:02