I am looking at the errors of my model, i.e. difference between predicted outputs and actual values. Finding a mean and standard deviation, I found that many values (sometimes more than 50%) are outside of 95% confidence interval. Does this mean that my error is no normally distributed? Is that even possible? Shouldn't 95% CI mean 95% of my values be in this range?
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
It doesn't mean that there isn't a problem, but you are comparing apples with oranges. The confidence interval is for the mean -- not the population. With a huge amount of data, the confidence interval for the mean will be very narrow because you can estimate the mean very accurately -- but almost all the data values will be outside that confidence interval.
Put another way, the confidence limits are about $\pm 2\sigma/\sqrt{n}$, while the 95% limits for a normal population are about $\pm 2\sigma$, without dividing by $\sqrt{n}$.
$\begingroup$
$\endgroup$
Here is a picture that shows what @rvl said. With reasonably large sample sizes, a tiny fraction of the values are within the 95% confidence interval of the mean. Source
.
answered Nov 24, 2014 at 1:02