# Variance between two coefficients in linear regression model

I am trying to figure out the variance of the difference between two coefficients in a linear regression model. If I am given the design matrix (X^T X) and the value of sigma, how do I go about solving: var(beta.1.hat - beta.2.hat)? My thought process is finding the variance for each part using the formula var(beta.j.hat) = sigma^2((X^T X)^-1 subscript jj. Then var(beta.1.hat - beta.2.hat) should be equal to: var(beta.1.hat) + var(beta.2.hat) - 2 Cov(beta.1.hat,beta.2.hat)

Is my logic correct? Please give me some suggestions or assistance thanks!

• If you know the variance of $Y$, then you could multiply $\beta$ by a matrix $[1, -1]$ and apply the same to the least squares estimator and calculate the variance as such.
– user275978
Mar 8, 2020 at 0:21

Assuming you have 2 variables, write $$\beta = (\beta_1, \beta_2)^T$$ and let $$c = (1, -1)^T$$. Recall that in ordinary least squares, $$Var(\hat\beta) = \sigma^2 (X^TX)^{-1}$$. \begin{align*} Var(\hat \beta_1 - \hat \beta_2) &= Var(c^T\hat\beta)\\ &= c^TVar(\hat\beta)c\\ &=\sigma^2 c^T(X^TX)^{-1}c \end{align*} This reduces to what you have, so your logic is correct.
If you don't remember the variance (or other parts of least squars. Posit model $$Y=X\beta + \epsilon$$ where $$\epsilon \sim (0, \sigma^2I)$$. Minimize the following objective function: \begin{align*} f(\beta) &= (Y-X\beta)^T(Y-X\beta)\\ f'(\beta) &= -2X^T(Y-X\beta) = 0\\ \hat\beta &= (X^TX)^{-1}X^TY\\ Var(\hat\beta) &= Var((X^TX)^{-1}X^TY)\\ &= (X^TX)^{-1}X^T Var(Y) X(X^TX)^{-1}\\ &= (X^TX)^{-1}X^T Var(X\beta + \epsilon) X(X^TX)^{-1}\\ &= (X^TX)^{-1}X^T Var(\epsilon) X(X^TX)^{-1}\\ &= (X^TX)^{-1}X^T \cdot \sigma^2I \cdot X(X^TX)^{-1}\\ &= \sigma^2(X^TX)^{-1} \end{align*} Note: normality assumption is not required unless you need to estimate $$\sigma^2$$ and/or do inference.