-1
$\begingroup$

Given this basic model:

$$ Y = \beta_0 + \beta_1 X + \varepsilon $$

one could think that predicting $Y$ for unseen values of $X$ is simply a question of plugging the $X$ values into this formula with the fitted parameters intercept and slope:

$$ Y = \beta_0 + \beta_1 X $$

However, I have read somewhere (cannot find the page despite many Google searches), that one has to consider the SD against the training data to simulate the Error during prediction. Is this correct? If so, what are the exact mechanics of this gain?

$\endgroup$

1 Answer 1

5
$\begingroup$

Do you want to make predictions using your model, or do you want to conduct a simulation?

The simple linear regression model is

$$ Y = \beta_0 + \beta_1 X + \varepsilon $$

and what we estimate is the conditional expected value

$$ E(Y|X) = \beta_0 + \beta_1 X $$

assuming that $Y|X$ is normally distributed with $\mu = \beta_0 + \beta_1 X$ and some standard deviation $\sigma$.

If you want to make predictions from your model, then given new value $x^*$ as your prediction for $y^*$ you take

$$ y^* = \hat\beta_0 + \hat\beta_1 x^* $$

so you predict what you were estimating, i.e. the conditional mean.

On another hand, if you want to simulate data according to your model, then you would draw samples from the normal distribution

$$ y_\text{sim} \sim \mathcal{N}(\hat\beta_0 + \hat\beta_1 x^*, \, \sigma) $$

The difference is that simulated values account for the "noise" $\varepsilon$. "Noise" is something that is unknown to you, it is random around the mean, so you do not include it in your predictions. You include the noise in your simulation, since when simulating, you want to imitate the randomness related to your model. So when predicting, you answer the question "according to your best knowledge, what will happen?" and when simulating, you answer the question "what could possibly happen in any circumstances?", those are different questions.

As image is worth thousand words, the following plot shows predictions from the linear regression model (red dots) against the data that was used to estimate the model (blue dots) and the values simulated according to the the model (gray dots).

Regression: data vs fitted values vs simulated values

$\endgroup$
5
  • $\begingroup$ Thanks do predictions can be made as I said. Good point about the simulation - although I am not 100% sure about the distinction. Would it not be a more realistic prediction to use a simulation (-: $\endgroup$
    – cs0815
    Commented Aug 16, 2017 at 15:35
  • $\begingroup$ @csetzkorn I already did, please check my edit. $\endgroup$
    – Tim
    Commented Aug 16, 2017 at 15:43
  • 2
    $\begingroup$ Realistic, yes--but good, no. This can be demonstrated by supposing that the discrepancy between the true value $x\beta+\epsilon$ and your prediction will incur a cost that increases with the size of the difference. When that is the case, adding a random term $\epsilon$ to your prediction $x\beta$ can only increase the expected cost. In short, if you have to guess an unknown number, you're better off guessing near an average of the possibilities rather than trying to guess a specific number randomly among the possibilities. $\endgroup$
    – whuber
    Commented Aug 16, 2017 at 15:46
  • $\begingroup$ @whuber I'm not sure if part of your comment was not lost since while I understand what do you mean, I cannot grasp the context. What did you meant to say? $\endgroup$
    – Tim
    Commented Aug 16, 2017 at 15:49
  • $\begingroup$ Tim, it's a reply to csetzkorn's query, "Would it not be a more realistic prediction to use a simulation." Your comment appeared while I was writing it, but I was not responding to you. $\endgroup$
    – whuber
    Commented Aug 16, 2017 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.