# Linear regression intercept proofs

Consider that $b_{0} = Y_1 - b_1X_1$ where $Y_1$ and $X_1$ here are the mean values for each. How would I derive the variance and other properties for $b_0$?

I'm not very good at doing proofs in statistics. Thanks in advance.

Disclaimer: I realize now, that you might be looking for a sketch of a simpler proof. I thought about not submitting this (partial) proof as an answer, but I guess someone else might find it useful. I'll try to add a simpler proof for the K=2 case (simple linear regression) later, if someone else doesn't do it.

## How to derive the variance for the OLS parameter estimates

There are many ways to derive the variance-covariance matrix for the problem you are stating. What you are actually missing are explicit assumptions about your model. Since you have not stated anything, I will make a harsh but popular assumption: homoscedasticity of the error term: $$E(u^2x'x)=\sigma^2E(x'x)$$ Given this assumption (along with your standard assumptions: linearity, random sample, population orthogonality $E(x'u)=0$ and rank $E(x'x)=K$) you can derive the variance-covariance matrix for $\beta=(\beta_0,\beta_1)$.

There are many ways to do this, and this is just one. It is as presented in Wooldridge (2010), but you could find variations of this depending on your textbook. First off let os consider the following: $$\sqrt{N}(\hat\beta-\beta)=\left(N^{-1}\sum^N_{i=1}x_i'x_i^{\,}\right)^{-1}\left(N^{-1/2}\sum^N_{i=1}x_i'u_i^{\,}\right)$$ If you can derive $\hat\beta$ you should be able to show this result. We are not quite done, though. There are two things to consider here. The first factor can be shown to fulfill the following: $$\left(N^{-1}\sum^N_{i=1}x_i'x_i^{\,}\right)^{-1}-A^{-1}=o_p (1)$$ where $A=E(x'x)$. For the second factor we have that the sequence $(x'_iu_i^{\,})_{i=1,...,N}$ is i.i.d. with mean zero. If we assume each element has finite variance, a Central Limit Theorem applies and thus: $$U=N^{-1/2}\sum^N_{i=1}x_i'u_i^{\,}\overset{d}{\to} \text{Normal}(0,E(u^2x'x)).$$ Where are we then? Basically we can express $\sqrt{N}(\hat\beta-\beta)$ as a product of a non-random matrix $A^{-1}$ and a factor we know the assymptotic distribution of ($U$), because: $$\sqrt{N}(\hat\beta-\beta)=\left(N^{-1}\sum^N_{i=1}x_i'x_i^{\,}\right)^{-1}\left(N^{-1/2}\sum^N_{i=1}x_i'u_i^{\,}\right) \\ =(A^{-1}+o_p (1))\left(N^{-1/2}\sum^N_{i=1}x_i'u_i^{\,}\right) \\ =A^{-1}\left(N^{-1/2}\sum^N_{i=1}x_i'u_i^{\,}\right)+o_p(1)\cdot O_p(1) \\ =A^{-1}\left(N^{-1/2}\sum^N_{i=1}x_i'u_i^{\,}\right)+o_p(1)\\ =A^{-1}U+o_p(1)$$ Now remember $A=E(x'x)$, our homoskedasticity assumption and our assymptotic distribution of U. These three things (and the above derivation) give us: $$\sqrt{N}(\hat\beta-\beta)\overset{a}{\sim} \text{Normal}(0,E(x'x)^{-1}\sigma^2 E(x'x) E(x'x)^{-1})\\ \sqrt{N}(\hat\beta-\beta)\overset{a}{\sim} \text{Normal}(0,E(x'x)^{-1}\sigma^2)$$ You can find your $Avar(\hat\beta)$ from this. In your case you will end up with a 2x2 diagonal matrix, and your variance estimate for the first parameter will be the first diagonal element.

Start by assuming all the classical OLS properties to hold, in addition to a stochastic interpretation of your RHS variables. First, simply take the variance of your expression (notice the conditioning on $X$)*: \begin{align} {\rm Var}(b_0|X) &= {\rm Var}(Y_1 -b_1 X_1|X) \\ &= {\rm Var}(Y_1|X)+X_{1}^{2}{\rm Var}(b_1|X)-2X_1{\rm Cov}(Y_1,b_1|X) \end{align} Now plug in

${\rm Var}(Y_1|X)=\frac{\sigma^2}{n}$ (if you have assumed IID data)

${\rm Var}(b_1|X)={\rm Var}(M_{XY}/M_{XX}|X)=\frac{\sigma^2}{n\hat{\sigma}_{X}^2}$ (Proof is straightforward, but cumbersome in notation, so I'm omitting it since you asked about $b_0$)

${\rm Cov}(Y_1,b_1|X)=0$. The only proof I know of this is somewhat long. Maybe someone else knows a short and handy one?

Combining this, you get: ${\rm Var}(b_0|X)=\frac{\sigma^2}{n}+X_{1}^{2}\frac{\sigma^2}{n\hat{\sigma}_{X}^2}$

*I assume you are interested in the $\hat{b}_0$, so please read the above as such.