7
$\begingroup$

The standard error of prediction in simple linear regression is $\hat\sigma\sqrt{1/n+(x_j-\bar{x})^2/\Sigma{(x_i-\bar{x})^2}}$.

My question is to calculate the standard error of prediction for $pop=1029$ just based on the following regression output. I can get all except $\bar{x}$. And I also know how to calculate the approximate standard error of prediction based on the standard errors of intercept and coefficient of $pop$, ignoring their correlation.

Stata regression output

$\endgroup$
4
  • 1
    $\begingroup$ How would the regression output change if you were, say, to add $10^6$ to each pop value and add $-0.0116584\times 10^6$ to each fuel value? Intuitively, that shifts the data far from pop=1029 without altering the regression line and therefore should result in a much wider prediction interval. That means you can focus your research on those elements of the output that change. (Even if you don't have the actual data you can make some up and run both regressions to see what happens.) $\endgroup$
    – whuber
    Jul 11, 2013 at 19:32
  • 1
    $\begingroup$ Thanks very much! Only the standard error of the intercept (therefore t, p-value and CI) changes. This inspired me to figure out that $Var(\hat{\beta}_0)=\sigma^2(1/n+\bar{x}^2/SXX)$, then I can get $\bar{x}$ to calculate the standard error of prediction. $\endgroup$
    – Randel
    Jul 11, 2013 at 20:39
  • 1
    $\begingroup$ The standard error of a predicted value isn't what you said. What you have there is the standard error for the mean at a given $x$. $\endgroup$
    – Glen_b
    Jul 12, 2013 at 2:41
  • $\begingroup$ Sorry I just followed the description of the option stdp in Stata. It can be thought of as the standard error of the predicted expected value, mean or the fitted value. $\endgroup$
    – Randel
    Jul 12, 2013 at 13:22

3 Answers 3

2
$\begingroup$

The question is to calculate the following statistic from the above regression output:

$$s.e.(\hat\mu|x_j)=\hat\sigma\sqrt{1/n+(x_j-\bar{x})^2/\Sigma{(x_i-\bar{x})^2}}.$$

The answer is inspired by @whuber:

  • get $\hat\sigma$ from $\hat\sigma^2=SS_{Residual}/(n-p-1)$, where $p=1$;
  • $n$ and $x_j$ are known;
  • obtain $\bar{x}$ from $\hat{Var}(β_{cons})=\hatσ^2(1/n+\bar{x}^2/\Sigma{(x_i-\bar{x})^2})$;
  • $\Sigma{(x_i-\bar{x})^2}=SS_{Model}/\hat{\beta}_{pop}^2$.
$\endgroup$
1
$\begingroup$

For simplicity, we are working with the following model: $$y=\beta_0 + \beta_1x + \varepsilon,$$ where $\varepsilon\sim N(0, \sigma^2)$.

Now suppose that for $x=5$, we would like to predict $E(y|x=5) = \beta_0 + 5\beta_1$, denoted by $pre$.

We note that $pre$ is just a value if we know $\beta_0$ and $\beta_1$. However, they are unknow and each of them has its own sampling distribution (where the sd of this distribution is called the se). Therefore, the standard error associated with $pre$ is computed as: $$Var(pre) = Var(\beta_0 + 5\beta_1) = Var(\beta_0) + 10 Cov(\beta_0, \beta_1) + 25Var(\beta_1).$$ Now you can see that you cannot compute this variance if you do not know the covariance $Cov(\beta_0, \beta_1)$.

$\endgroup$
0
$\begingroup$

See Section 6-4a of Wooldridge (2020), Introductory Econometrics: A Modern Approach, 7ed, Cengage.

Variance of prediction error

It's not $\sigma^2 [ 1/n + (x_j-\bar{x})^2 / \sum_i (x_i - \bar{x})^2 ]$.

For the correct expression, let me use vector notations. The model is $y=x\beta + u$, where $x=(1,pop)$. Let $x_0 = (1,1029)$. The value to predict ($y_0$) is not yet labeled. The predictor is $\hat\theta = x_0 \hat\beta$, where $\hat\beta$ is the OLS estimator, and the label to predict is $x_0 \beta + u_0$. The prediction error is thus $y_0 - \hat\theta = u_0 - x_0 (\hat\beta-\beta)$.

As $\hat\beta$ is a function of the sample and $u_0$ (out of sample) is assumed to be independent of the sample, the variance of the prediction error is $\sigma^2 + x_0 V(\hat\beta) x_0'$, where $'$ stands for transpose. As $V(\hat\beta) = \sigma^2 (X'X)^{-1}$, where $X$ is the feature matrix (including the constant term in the first column), the variance of the prediction error is $\sigma^2 [ 1+ x_0 (X'X)^{-1} x_0' ]$ so the standard error is $$\sigma \sqrt{1 + x_0 (X'X)^{-1} x_0'}.$$

Calculation of se(prediction error)

If we change the variable $pop$ to $pop-1029$, then $\hat\theta$ is reported as the intercept estimate:

/* Stata */
gen pop2 = pop - 1029
reg fuel pop2

You see that the reparameterizated model $fuel = \theta + \beta_1 (pop-1029) + u$ gives $\theta = \beta_0 + 1029 \beta_1$ so the intercept estimator $\hat\theta$ is your predictor.

The reported standard error is, however, $se(\hat\theta) = \hat\sigma \sqrt{x_0 (X'X)^{-1} x_0'}$, not the desired $\hat\sigma \sqrt{1+x_0 (X'X)^{-1} x_0'}$. For the correct one, you can simply compute $\sqrt{\hat\sigma^2 + se(\hat\theta)^2}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.