Distribution of linear combination of OLS regression coefficients

Question

I have a simple linear OLS regression $Y_i = \alpha+ \beta_1 X_{1i} + \beta_2 X_{2i} + e_i$ where $e_i \sim N(0,\sigma)$. I have estimated the regression from the data and obtained estimates for my coefficients as well as the corresponding covariance matrix. Assume my dataset is large (>500)

Now I would like to construct the 95% confidence interval for $Z = a_1 \beta_1 + a_2 \beta_2$. My first thought was that I could assume that Z follows a normal distribution because my regression coefficients are approximately normal and a linear combination of normal variables is also a normal variable. I do find some sources where they treat $Z$ as a normal random variable.

But now I am in doubt because I found that a linear combination of normal random variables is only normal if they are independent. So is it still save to assume $Z$ is random?

Answer 1

It's actually not true that your variables need to be independent so that their sum will be normal.

If X and Y are jointly normally distributed with mean $\mu_{1} $ and $\mu_{2} $ and variance $\sigma_{1}^{2}$ and $\sigma_{2}^{2}$ with correlation $\rho $ then Z is still normally distributed with mean $\mu_{1}+\mu_{2} $ and variance $\sqrt { \sigma_{2}^{2}+ \sigma_{1}^{2}+2 \rho \sigma_{2} \sigma_{1}}$ Hopefully that gives you what you need.

Answer 2

it would be better to explain this in matrix notation. Suppose the general Gauss-Markov linear model $$\mathbf{y = X \boldsymbol \beta + \boldsymbol \epsilon}$$For your case, $\mathbf{X}$ = ($\mathbf{1}$, $\mathbf{x_1}$, $\mathbf{x_2}$) and $\mathbf{\boldsymbol \beta} = (\alpha, \; \beta_1 \; \beta_2)'.$ The OLS estimator is $$\hat{\boldsymbol \beta} = \left(\mathbf{X}'\mathbf{X} \right)^{-1}\mathbf{X}'\mathbf{y}.$$ Since $\mathbf{y} \ \sim \ N(\mathbf{X \boldsymbol \beta}, \sigma^2)$, then $$\hat{\boldsymbol \beta} \ \sim \ N \left(\mathbf{X \boldsymbol \beta}, \sigma^2(\mathbf{X}'\mathbf{X})^{-1} \right).$$ Based on the above result, you are able to derive the distribution of $Z = \alpha_1 \beta_1 + \alpha_2 \beta_2$ if you write $Z = \boldsymbol \lambda' \boldsymbol \beta$, where $\lambda' = (0, \alpha_1, \alpha_2)$. Denote correspondingly that $$\hat{Z} = \alpha_1 \hat{\beta}_1 + \alpha_2 \hat{\beta}_2 = \boldsymbol \lambda' \hat{\boldsymbol \beta},$$ then you can easily get that $$\frac{\hat{Z} - Z}{\sigma\sqrt{\boldsymbol \lambda' (\mathbf{X}' \mathbf{X})^{-1} \boldsymbol \lambda}} \ \sim \ N(0, 1).$$ Note that $\sigma$ is unknown, so you cannot construct the confidence interval based on the normal distribution. A rigorous approach is to derive the confidence interval based on a $t$-test statistic (I just ignore the derivation but give the result, you should be able to find details in any linear model book or through website). Specifically,$$\frac{\hat{Z} - Z}{\hat{\sigma} \sqrt{\boldsymbol \lambda' (\mathbf{X}' \mathbf{X})^{-1} \boldsymbol \lambda}} \ \sim \ t(n-p),$$ where $\hat{\sigma}^2 = \frac{\mathbf{y}'(\mathbf{I - P_X}) \mathbf{y}}{n-p}$, $\mathbf{P_X}$ is the projection matrix, and $p = rank(\mathbf{X})$.