9
$\begingroup$

Suppose I have a quadratic regression model $$ Y = \beta_0 + \beta_1 X + \beta_2 X^2 + \epsilon $$ with the errors $\epsilon$ satisfying the usual assumptions (independent, normal, independent of the $X$ values). Let $b_0, b_1, b_2$ be the least squares estimates.

I have two new $X$ values $x_1$ and $x_2$, and I'm interested in getting a confidence interval for $v = E(Y|X = x_2) - E(Y|X=x_1) = \beta_1 (x_2 - x_1) + \beta_2 (x_2^2 - x_1^2)$.

The point estimate is $\hat{v} = b_1 (x_2 - x_1) + b_2 (x_2^2 - x_1^2)$, and (correct me if I'm wrong) I can estimate the variance by $$\hat{s}^2 = (x_2 - x_1)^2 \text{Var}(b_1) + (x_2^2 - x_1^2)^2 \text{Var}(b_2) + 2 (x_2 - x_1)(x^2 - x_1^2)\text{Cov}(b_1, b_2)$$ using the variance and covariance estimates of the coefficients provided by the software.

I could use a normal approximation and take $\hat{v} \pm 1.96 \hat{s}$ as a 95% confidence interval for $v$, or I could use a bootstrap confidence interval, but is there a way to work out the exact distribution and use that?

$\endgroup$
  • 2
    $\begingroup$ Because the errors are assumed normal, then the parameter estimates--being linear functions of the data, whence of the errors too--must themselves be normal, implying a normal distribution for $\hat{v}$. $\endgroup$ – whuber May 6 '11 at 21:52
  • $\begingroup$ So are you saying that the normal confidence interval is correct? If I understand correctly, by that logic we would also use normal confidence intervals for the parameters. But we use intervals based on the t distribution. $\endgroup$ – markseeto May 7 '11 at 3:38
  • $\begingroup$ The t distribution is used because you're estimating the error variance; if that were known then you would have a normal distribution like @whuber says. $\endgroup$ – JMS May 7 '11 at 5:39
  • $\begingroup$ Thanks for your comment. What I'm asking is, can the t distribution also be used for a confidence interval for v as defined in the question, and if so, with how many degrees of freedom? $\endgroup$ – markseeto May 7 '11 at 6:05
  • $\begingroup$ The variances and covariances all ultimately depend on the estimated variance of the residuals. Thus the DF to use is the DF in this estimate, equal to the number of data values minus the number of parameters (including the constant). $\endgroup$ – whuber May 7 '11 at 17:44
8
$\begingroup$

The general result you are looking for (under the stated assumptions) looks like this: For linear regression with $p$ predictor variables (you have two, $X$ and $X^2$) and an intercept, then with $n$ observations, $\mathbf{X}$ the $n \times (p+1)$ design matrix, $\hat{\beta}$ the $p+1$ dimensional estimator and $a \in \mathbb{R}^{p+1}$

$$ \frac{a^T\hat{\beta} - a^T \beta}{\hat{\sigma} \sqrt{a^T(\mathbf{X}^T\mathbf{X})^{-1}a}} \sim t_{n-p-1}.$$

The consequence is that you can construct confidence intervals for any linear combination of the $\beta$ vector using the same $t$-distribution you use to construct a confidence interval for one of the coordinates.

In your case, $p = 2$ and $a^T = (0, x_2 - x_1, x_2^2 - x_1^2)$. The denominator in the formula above is the square root of what you compute as the estimate of the standard error (provided that this is what the software computes ...). Note that the variance estimator, $\hat{\sigma}^2$, is supposed to be the (usual) unbiased estimator, where you divide by the degrees of freedom, $n-p-1$, and not the number of observations $n$.

$\endgroup$
  • 1
    $\begingroup$ Thank you, that is exactly the sort of thing I was looking for. But is there a mistake in the formula? The dimensions don't seem to match in $a^T(\mathbf{X}^T\mathbf{X})^{-1}a$. Should $\mathbf{X}$ be the $n \times (p+1)$ matrix having ones in the first column? $\endgroup$ – markseeto May 7 '11 at 9:39
  • $\begingroup$ @mark999, yes, $\mathbf{X}$ has $p+1$ columns. I have corrected that in the answer. Thanks. $\endgroup$ – NRH May 7 '11 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.