5
$\begingroup$

I am looking at the errors of my model, i.e. difference between predicted outputs and actual values. Finding a mean and standard deviation, I found that many values (sometimes more than 50%) are outside of 95% confidence interval. Does this mean that my error is no normally distributed? Is that even possible? Shouldn't 95% CI mean 95% of my values be in this range?

$\endgroup$
11
$\begingroup$

It doesn't mean that there isn't a problem, but you are comparing apples with oranges. The confidence interval is for the mean -- not the population. With a huge amount of data, the confidence interval for the mean will be very narrow because you can estimate the mean very accurately -- but almost all the data values will be outside that confidence interval.

Put another way, the confidence limits are about $\pm 2\sigma/\sqrt{n}$, while the 95% limits for a normal population are about $\pm 2\sigma$, without dividing by $\sqrt{n}$.

$\endgroup$
4
$\begingroup$

Here is a picture that shows what @rvl said. With reasonably large sample sizes, a tiny fraction of the values are within the 95% confidence interval of the mean. Source

. enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.