I have a simple linear OLS regression $Y_i = \alpha+ \beta_1 X_{1i} + \beta_2 X_{2i} + e_i$ where $e_i \sim N(0,\sigma)$. I have estimated the regression from the data and obtained estimates for my coefficients as well as the corresponding covariance matrix. Assume my dataset is large (>500)

Now I would like to construct the 95% confidence interval for $Z = a_1 \beta_1 + a_2 \beta_2$. My first thought was that I could assume that Z follows a normal distribution because my regression coefficients are approximately normal and a linear combination of normal variables is also a normal variable. I do find some sources where they treat $Z$ as a normal random variable.

But now I am in doubt because I found that a linear combination of normal random variables is only normal if they are independent. So is it still save to assume $Z$ is random?

  • 2
    $\begingroup$ Your coefficients are constants, so I wonder what you mean by saying that they "are approximately normal". At any rate, your coefficients are a function of your data (viz. your $Y$ values), which are random. Thus, they are random, & functions of them ($Z$) are random in turn. I think you meant to ask if it is safe to assume they are independent. This will turn out to be true if $X_1$ & $X_2$ are independent. However, a linear combination of normal random variables is still random if the variables are not independent, you just have to use a more complicated formula. $\endgroup$ Jan 13, 2015 at 22:50
  • $\begingroup$ I actually meant the sampling distribution of the estimates. I actually want to construct the 95% CI of $z = \beta_1 + \beta_2$. My approach was to determine the expected value as the sum of the estimates and the variance of z with the formula of the variance for correlated variables ($\operatorname{Var}\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \sum_{j=1}^n \operatorname{Cov}(X_i, X_j) = \sum_{i=1}^n \operatorname{Var}(X_i) + 2\sum_{1\le i}\sum_{<j\le n}\operatorname{Cov}(X_i,X_j)$. In order to construct a CI, I need to know if the distribution of Z is normal. $\endgroup$
    – user58571
    Jan 13, 2015 at 23:03
  • $\begingroup$ Related: stats.stackexchange.com/questions/16724/… $\endgroup$
    – Andrew M
    Jan 13, 2015 at 23:13

2 Answers 2


It's actually not true that your variables need to be independent so that their sum will be normal.

If X and Y are jointly normally distributed with mean $\mu_{1} $ and $\mu_{2} $ and variance $\sigma_{1}^{2}$ and $\sigma_{2}^{2}$ with correlation $\rho $ then Z is still normally distributed with mean $\mu_{1}+\mu_{2} $ and variance $\sqrt { \sigma_{2}^{2}+ \sigma_{1}^{2}+2 \rho \sigma_{2} \sigma_{1}}$ Hopefully that gives you what you need.

  • $\begingroup$ This is not true. Let X be N(0,1) and Y be exactly -X. Clearly Y is also N(0,1), but X+Y = 0 almost surely, which is NOT normal. $\endgroup$ Jan 13, 2015 at 22:51
  • 1
    $\begingroup$ Sorry, I should have included that they should be jointly normal. $\endgroup$ Jan 13, 2015 at 22:51
  • 1
    $\begingroup$ A necessary and sufficient condition is that their Covariance matrix is positive definite. That prevents my counterexample and supports your claim. $\endgroup$ Jan 13, 2015 at 22:54
  • $\begingroup$ Ok, but then the question remains if one can assume that the regression coefficients of an OLS regression are jointly normally distributed? If so, any pointers to some source which derives/explains this assumption would be great $\endgroup$
    – user58571
    Jan 13, 2015 at 23:09
  • 1
    $\begingroup$ @ChristopherAden A necessary condition of OLS is that the covariance matrix is positive definite, if this were not true you could not even estimate the coefficients. Thus under the assumptions of OLS where the error are i.i.d normal, any linear combinations of estimated coefficients is also normal with mean and variance calculated similar to as suggested above. $\endgroup$ Jan 13, 2015 at 23:10

it would be better to explain this in matrix notation. Suppose the general Gauss-Markov linear model $$\mathbf{y = X \boldsymbol \beta + \boldsymbol \epsilon}$$For your case, $\mathbf{X}$ = ($\mathbf{1}$, $\mathbf{x_1}$, $\mathbf{x_2}$) and $\mathbf{\boldsymbol \beta} = (\alpha, \; \beta_1 \; \beta_2)'.$ The OLS estimator is $$\hat{\boldsymbol \beta} = \left(\mathbf{X}'\mathbf{X} \right)^{-1}\mathbf{X}'\mathbf{y}.$$ Since $\mathbf{y} \ \sim \ N(\mathbf{X \boldsymbol \beta}, \sigma^2)$, then $$\hat{\boldsymbol \beta} \ \sim \ N \left(\mathbf{X \boldsymbol \beta}, \sigma^2(\mathbf{X}'\mathbf{X})^{-1} \right).$$ Based on the above result, you are able to derive the distribution of $Z = \alpha_1 \beta_1 + \alpha_2 \beta_2$ if you write $Z = \boldsymbol \lambda' \boldsymbol \beta$, where $\lambda' = (0, \alpha_1, \alpha_2)$. Denote correspondingly that $$\hat{Z} = \alpha_1 \hat{\beta}_1 + \alpha_2 \hat{\beta}_2 = \boldsymbol \lambda' \hat{\boldsymbol \beta},$$ then you can easily get that $$\frac{\hat{Z} - Z}{\sigma\sqrt{\boldsymbol \lambda' (\mathbf{X}' \mathbf{X})^{-1} \boldsymbol \lambda}} \ \sim \ N(0, 1).$$ Note that $\sigma$ is unknown, so you cannot construct the confidence interval based on the normal distribution. A rigorous approach is to derive the confidence interval based on a $t$-test statistic (I just ignore the derivation but give the result, you should be able to find details in any linear model book or through website). Specifically,$$\frac{\hat{Z} - Z}{\hat{\sigma} \sqrt{\boldsymbol \lambda' (\mathbf{X}' \mathbf{X})^{-1} \boldsymbol \lambda}} \ \sim \ t(n-p),$$ where $\hat{\sigma}^2 = \frac{\mathbf{y}'(\mathbf{I - P_X}) \mathbf{y}}{n-p}$, $\mathbf{P_X}$ is the projection matrix, and $p = rank(\mathbf{X})$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.