# Meaning of “standard deviation units” in a test for statistical significance (basic)

For my chemistry class, I have to do some statistics on a set of data. One of the things I have to do is eliminate outliers via the 4D rule. Unfortunately my teacher is also absent, so I can't get this question answered.

The 4D rule goes like this on my stat sheet:

1. Calculate mean and standard deviation.

2. Establish a range of acceptable results by adding and subtracting two standard deviation units to/from the mean.

3. Calculate new mean and standard deviation after excluding the results that are out of range.

4. Multiply new standard deviation by four and calculate the difference between each of the original results (without exclusions) and the new mean.

5. If the differences calculated is greater than 4 * standard deviation (calculated in step 4), it is valid to exclude the trial.

And my data points are: 1.1, 4.5, 9.4, 9.7, 9.9, 10.2, 22.2. (they are rounded short for this post)

Mean: 9.6, standard deviation: 5.3

Is it correct to then say that the acceptable range is -1 to 20.2. So I can only exclude the last data point and not the first two?

Would the results be difference if I changed the measurements from micrometers to nanometers then?

-
 By the way, although it doesn't affect the result, I think you miscalculated the standard deviation. – Douglas Zare Aug 26 '12 at 9:23 I used Excel to do it. I used STDEV.P() instead of STDEV(), so, while I don't have the sheet in front of me, I think the correct value is about 5.6. – Eric Thoma Aug 26 '12 at 23:38 That's still not the value I calculated. – Douglas Zare Aug 27 '12 at 3:53

 If you have normally distributed data, I don't think very much of it will be excluded by this procedure, since most will be added back after it falls within $4$ standard deviations of the mean of the reduced sample. Outliers are a real problem, and I don't see a problem with learning a procedure for removing them. It is on non-normal data where this procedure has a danger of removing too many points. – Douglas Zare Aug 26 '12 at 9:22 @DouglasZare I didn't think very much about what would happen at the second stage when you can add observations back. So it may be that some of the less extreme outliers will be brought back. Regardng methods for identifying outliers, I agree that it is important to identify outliers but removing them should not be automatic. Also what are the statistical properties of this procedure that makes it desirable. For normally distributed data Grubbs' test and Dixon's ratio test are well-known methods for identifying one or a few outlying observations with desirable properties. So why elect this? – Michael Chernick Aug 26 '12 at 11:44 It seems like a very ad hoc procedure. – Michael Chernick Aug 26 '12 at 11:44 It doesn't look much different from Grubb's test or Dixon's test to me. It has the virtue of being simpler, as though one has chosen a particular significance level. If it's presented as a critical procedure to do all of the time, that's not good, but if it was accompanied by a brief discussion of outliers, why they occur, why you might want to get rid of them, and the dangers of doing so, then it seems like a good thing to include in a chemistry class. I wish my science classes had been as thorough. – Douglas Zare Aug 26 '12 at 19:01 The semester is really young, but we haven't really discussed it at all. The handout with it explained though that there was no 'perfect' method for outliers, and so the method strived to remove results sparingly. It did not mention though the points that have been brought up here. I think it is just making the assumption that the results should be uniform. In this case we were measuring the thickness of zinc on galvanized iron, so anomalies should be eliminated. No one has been through a statistics class in my AP chem class. – Eric Thoma Aug 26 '12 at 23:44