95% confidence interval for a given data set I have a data set $\mathcal{S}$, which is univariate. I would like to discover the data points which can be thought of as outliers of $\mathcal{S}$. I use   $\bar{\mathcal{S}}+2*\sigma(\mathcal{S})$ and $\bar{\mathcal{S}}-2*\sigma(\mathcal{S})$ to compute the upper and lower threshold in terms of 95% confidence interval. $\bar{\mathcal{S}}$ is the mean of $\mathcal{S}$ and $\sigma(\mathcal{S})$ is the standard deviation. I know it assumes the normal distribution. I would like to know are there any loopholes for this approach from the statistical analysis perspective. Moreover, are there any other better statistical approaches to give this kind of threshold.
 A: A few comments:


*

*Your title "95% confidence interval for a given data set" is misleading. Confidence intervals are calculated for computed values like means. It doesn't mean anything really to have a confidence interval for a data set. 

*Your proposed method will work with huge data sets, but will fail with small data sets. Why? Because the outlier itself increases the SD (assuming you are computing the SD from your actual values). With small data sets (say n=4), it is impossible for any outlier to be more than 2 SD from the mean. Try it! The most SD that a value can be from the mean is (n-1)/sqrt(n), where n is sample size.

*There are established methods for determining outliers assuming the other values are sampled from a Gaussian distribution. Look up Grubbs test, for one of the most widely used tests.

*Deciding what to do with outliers is tricky. 
A: An outlier is a surprising value.
Therefore, your method is flawed.
If the data set is large, then, if it is normally distributed, you expect some values beyond 2 sd from the mean. If the data set is small, and normally distributed, you don't expect values even 1 sd from the mean. e.g. it is not surprising that there are some 7 foot tall people in the USA. 
In addition, you have calculated the mean and sd with the possible outliers IN the data. If the outliers are truly not part of the data (e.g. they are data entry errors; they got picked for the sample by mistake) then you should not have done this.
I would say that there are no really good statistical tests for outliers, even univariately. At best, there are ways to identify points that might be worth looking at.  A better way than the one you propose is to graph the data. If you suppose it is normal, you could do a normal quantile plot. Even if you don't suppose it is normal, you could do a density plot. 
In addition, you should always think of the substantive nature of the values. Is it *possible? E.g., among human adults, a weight of 500 pounds is possible (albeit very large). A weight of 5000 pounds is not. 
A: Another approach you might consider is Tukey’s Outlier Filter. It is robust as well, since it is based on quantiles. 
The idea behind is to classify as outlier data points that lies:


*

*above 3rd quartile + 1.5 times the Inter Quartile Range (IQR, distance between the 1st and  3rd quartiles), and

*below 1st quartile - 1.5 times the IQR.


The 1.5 range is arbitrary and was suggest empirically. You might use 3 instead to be more strict. Take care with asymmetric data, which would need an adjustment in the classification criterion. 
Boxplot is a good plot to evaluate the points that may be considered outliers through this filter. See R example below applied to data posted by user603:
x <- c(-2.21,-1.84,-.95,-.91,-.36,-.19,-.11,-.1,.18,.3,.31,.43,.51,.64,.67,.72,1.22,1.35,8.1,17.6)
boxplot(x, range=3)


Reading suggestions: 


*

*SEO, Songwon. "A review and comparison of methods for detecting outliers in univariate data sets" (http://d-scholarship.pitt.edu/7948/1/Seo.pdf)

*MASSART, Desire Luc et al. "Visual Presentation of Data by Means of Box Plots". (http://www.lcgceurope.com/lcgceurope/data/articlestandard/lcgceurope/132005/152912/article.pdf) 

