# Distribution of linear combination of OLS regression coefficients

I have a simple linear OLS regression $Y_i = \alpha+ \beta_1 X_{1i} + \beta_2 X_{2i} + e_i$ where $e_i \sim N(0,\sigma)$. I have estimated the regression from the data and obtained estimates for my coefficients as well as the corresponding covariance matrix. Assume my dataset is large (>500)

Now I would like to construct the 95% confidence interval for $Z = a_1 \beta_1 + a_2 \beta_2$. My first thought was that I could assume that Z follows a normal distribution because my regression coefficients are approximately normal and a linear combination of normal variables is also a normal variable. I do find some sources where they treat $Z$ as a normal random variable.

But now I am in doubt because I found that a linear combination of normal random variables is only normal if they are independent. So is it still save to assume $Z$ is random?

• Your coefficients are constants, so I wonder what you mean by saying that they "are approximately normal". At any rate, your coefficients are a function of your data (viz. your $Y$ values), which are random. Thus, they are random, & functions of them ($Z$) are random in turn. I think you meant to ask if it is safe to assume they are independent. This will turn out to be true if $X_1$ & $X_2$ are independent. However, a linear combination of normal random variables is still random if the variables are not independent, you just have to use a more complicated formula. – gung Jan 13 '15 at 22:50
• I actually meant the sampling distribution of the estimates. I actually want to construct the 95% CI of $z = \beta_1 + \beta_2$. My approach was to determine the expected value as the sum of the estimates and the variance of z with the formula of the variance for correlated variables ($\operatorname{Var}\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \sum_{j=1}^n \operatorname{Cov}(X_i, X_j) = \sum_{i=1}^n \operatorname{Var}(X_i) + 2\sum_{1\le i}\sum_{<j\le n}\operatorname{Cov}(X_i,X_j)$. In order to construct a CI, I need to know if the distribution of Z is normal. – user58571 Jan 13 '15 at 23:03
• – Andrew M Jan 13 '15 at 23:13

If X and Y are jointly normally distributed with mean $\mu_{1}$ and $\mu_{2}$ and variance $\sigma_{1}^{2}$ and $\sigma_{2}^{2}$ with correlation $\rho$ then Z is still normally distributed with mean $\mu_{1}+\mu_{2}$ and variance $\sqrt { \sigma_{2}^{2}+ \sigma_{1}^{2}+2 \rho \sigma_{2} \sigma_{1}}$ Hopefully that gives you what you need.
it would be better to explain this in matrix notation. Suppose the general Gauss-Markov linear model $$\mathbf{y = X \boldsymbol \beta + \boldsymbol \epsilon}$$For your case, $\mathbf{X}$ = ($\mathbf{1}$, $\mathbf{x_1}$, $\mathbf{x_2}$) and $\mathbf{\boldsymbol \beta} = (\alpha, \; \beta_1 \; \beta_2)'.$ The OLS estimator is $$\hat{\boldsymbol \beta} = \left(\mathbf{X}'\mathbf{X} \right)^{-1}\mathbf{X}'\mathbf{y}.$$ Since $\mathbf{y} \ \sim \ N(\mathbf{X \boldsymbol \beta}, \sigma^2)$, then $$\hat{\boldsymbol \beta} \ \sim \ N \left(\mathbf{X \boldsymbol \beta}, \sigma^2(\mathbf{X}'\mathbf{X})^{-1} \right).$$ Based on the above result, you are able to derive the distribution of $Z = \alpha_1 \beta_1 + \alpha_2 \beta_2$ if you write $Z = \boldsymbol \lambda' \boldsymbol \beta$, where $\lambda' = (0, \alpha_1, \alpha_2)$. Denote correspondingly that $$\hat{Z} = \alpha_1 \hat{\beta}_1 + \alpha_2 \hat{\beta}_2 = \boldsymbol \lambda' \hat{\boldsymbol \beta},$$ then you can easily get that $$\frac{\hat{Z} - Z}{\sigma\sqrt{\boldsymbol \lambda' (\mathbf{X}' \mathbf{X})^{-1} \boldsymbol \lambda}} \ \sim \ N(0, 1).$$ Note that $\sigma$ is unknown, so you cannot construct the confidence interval based on the normal distribution. A rigorous approach is to derive the confidence interval based on a $t$-test statistic (I just ignore the derivation but give the result, you should be able to find details in any linear model book or through website). Specifically,$$\frac{\hat{Z} - Z}{\hat{\sigma} \sqrt{\boldsymbol \lambda' (\mathbf{X}' \mathbf{X})^{-1} \boldsymbol \lambda}} \ \sim \ t(n-p),$$ where $\hat{\sigma}^2 = \frac{\mathbf{y}'(\mathbf{I - P_X}) \mathbf{y}}{n-p}$, $\mathbf{P_X}$ is the projection matrix, and $p = rank(\mathbf{X})$.