I did a simple linear regression of predicted values (by my model) vs. measured values and made confidence interval boundaries (95% confidence) of the fit using matlab functions. I know that a 95% confidence interval means that there is a 95% probability that the true best-fit line for the population lies within the confidence interval. But, what I wish to know is if there is anything significant about the points that lie outside of this boundary? Are they outliers of some sort? If I have a lot of points outside of the boundaries as compared to a few, does that necessarily mean I have a bad model? Thank you.

  • 1
    $\begingroup$ Have you studied what happens to the confidence bands as the number of points is increased? If not, doing so will be revealing. $\endgroup$ – whuber May 17 '17 at 23:03

Let's take a step back. The definition of the 95% confidence interval that you gave is not the correct one. In frequentist statistics, the true value is fixed and it will either be inside the interval or not. Your model's predicted values are random variables. Meaning that if someone else takes a different sample will have different predicted values and thus, different points in the graphs. This is why y hat is a random variable. A 95% CI means that: For every point in your graph, if 100 people take 100 samples we will have 100 CIs in total for each point. Your model "guesses" that 95 of these 100 intervals will contain the true value and this is the correct definition of a confidence interval.

That being said, if you have a lot of outliers that could be evidence that your model is not "guessing" very well and that there is variance in your response variable that is not adequately explained by your model. You may consider fitting a better one (there are numerous things you can do to enhance it. For instance, if you don't have additional predictors maybe consider adding the square of the independent variable in the model?)

Hope that helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.