# Does high number of values outside of 95% Confidence Interval imply non-normality?

I am looking at the errors of my model, i.e. difference between predicted outputs and actual values. Finding a mean and standard deviation, I found that many values (sometimes more than 50%) are outside of 95% confidence interval. Does this mean that my error is no normally distributed? Is that even possible? Shouldn't 95% CI mean 95% of my values be in this range?

It doesn't mean that there isn't a problem, but you are comparing apples with oranges. The confidence interval is for the mean -- not the population. With a huge amount of data, the confidence interval for the mean will be very narrow because you can estimate the mean very accurately -- but almost all the data values will be outside that confidence interval.

Put another way, the confidence limits are about $\pm 2\sigma/\sqrt{n}$, while the 95% limits for a normal population are about $\pm 2\sigma$, without dividing by $\sqrt{n}$.

Here is a picture that shows what @rvl said. With reasonably large sample sizes, a tiny fraction of the values are within the 95% confidence interval of the mean. Source

.