# Asymptotic variance of a single parameter in linear regression

Consider the linear regression model $$Y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \dots + \beta_k x_{ik} + \epsilon_i$$ or equivalently in matrix norm $$\mathbf{Y} = \beta \mathbf{X} + \epsilon$$ where $$\mathbb{E}[\epsilon|X] = 0$$, $$\mathbf{X}$$ is $$n \times (k + 1)$$ matrix. It can be shown by the Central Limit Theorem that the OLS estimates satisfy $$\sqrt{n}(\hat{\beta} - \beta) \to^d \mathcal{N}(0, \Sigma)$$ where $$\Sigma = \mathbb{E}[XX']^{-1} \mathbb{E}[XX'\epsilon^2] \mathbb{E}[XX']^{-1}$$ is a $$(k+1) \times (k+1)$$ matrix.

Suppose we were interested in inference only on $$\beta_1$$. Is there a way to analytically derive the asymptotic variance of only $$\hat{\beta}_1$$? More precisely, the expression for $$\sigma^2_{\beta_1}$$ that satisfies $$\sqrt{n}(\hat{\beta}_1 - \beta_1) \to^d \mathcal{N}(0, \sigma^2_{\beta_1})$$ I think it should be the $$(2,2)$$ entry of $$\Sigma$$, though I'm not sure how to separate it out analytically given the matrix inverses. Computationally, we can just take the relevant entry $$\hat{\Sigma}_{2,2}$$ from the full estimated variance matrix but this would be very inefficient (especially if $$k$$ large) since we only need a single entry. I was trying some ideas with the Frisch-Waugh-Lovell Theorem but not quite getting anywhere.

Any ideas?

• Unless you are assuming $X$ is stochastic, your expression for $\Sigma$ is unnecessarily complex, and in any case you have the transposes wrong; $\sigma^2_{\epsilon}(X'X)^{-1}$ is correct. Jan 21 at 16:50
• @jbowman, I agree about the transposes, but as to the complexity, the result should be the heteroskedasticity-robust one (and hence not so much related to whether or not regressors are stochastic), and since heteroskedasticity is quite pervasive in applied work I would not call it unnecessary to consider this expression. Jan 21 at 17:40
• @Adam, I would try it via partitioned inverses like referenced e.g. here: stats.stackexchange.com/questions/258461/… Not clear if you will obtain a "clean" expression Jan 21 at 17:42
• @ChristophHanck Thanks! I'll take a look at that - maybe it simplifies.
Jan 22 at 4:40
• I cannot see that there's any content to the question, so perhaps I'm misinterpreting it. Doesn't the asymptotic convergence in distribution to a multivariate Normal already tell you what the asymptotic convergence of any one of the estimates is?
– whuber
Feb 17 at 22:59

We may "need a single entry" only, but all the sample will participate in computing it. Write your model as $$Y_i = \beta_1 x_{i1} + \beta_0 + + \beta_2 x_{i2} + \dots + \beta_k x_{ik} + \epsilon_i$$ and partition the $$n \times k$$ regressore matrix as $$\mathbf X = \left [\mathbf x_1\quad Z \right],$$ where $$Z$$ contains all other regressors including the constant. Let $$D = {\rm diag} \{\hat \epsilon^2_i\}$$, a $$n \times n$$ diagonal matrix. Then, in practice, $$\widehat \Sigma = n\left [\begin{matrix} \mathbf x_1'\mathbf x_1 & \mathbf x_1'Z \\ Z'\mathbf x_1 & Z'Z \end{matrix}\right]^{-1}\left [\begin{matrix} \mathbf x_1'D\mathbf x_1 & \mathbf x_1'DZ \\ Z'D\mathbf x_1 & Z'DZ \end{matrix}\right]\left [\begin{matrix} \mathbf x_1'\mathbf x_1 & \mathbf x_1'Z \\ Z'\mathbf x_1 & Z'Z \end{matrix}\right]^{-1}.$$
The upper left element will be $$1 \times 1$$ and it is what you want.
Apply the most convenient matrix blockwise inversion formula, and obtain the final upper left element of $$\widehat \Sigma$$. Determine whether computing only it results in improvements as regards computational efficiency.
NOTE 1: You need to correct the expression for $$\Sigma$$ in your post (the expected value should be under the inverse sign).
NOTE 2: The $$n$$ in front of the expression for $$\widehat \Sigma$$ is because we compute the variance of $$\sqrt{n}(\hat{\beta} - \beta)$$. If we want the approximation for the finite sample variance of $$\hat \beta$$, we ignore it.