# How to check if removing a sample makes a difference in mean and stdev values?

I'd like to ask if someone could help me with the following problem:

we have measured the same sample 5 times and we would like to check if there are significant differences in mean and stdev values if we use:

• All 5 datapoints
• Only the last 4 datapoints
• Only the last 3 datapoints

We have performed ANOVA analysis but we are not sure about the results because we might not have homocedasticity.

Which tests would you do to investigate this issue?

• As @EngrStudent mentions below, 5 data points isn't much. But, could you perhaps describe your data a bit better? – Jason Morgan Apr 2 '13 at 17:30
• @JasonMorgan i suspect that with datapoint, he refers to a single time they measured the sample. Not to the number of observations within a single measuring session. – mvherweg Aug 1 '13 at 10:55

Not only don't you have much data (it's very unlikely to be enough to pick up any change even moving from 5 points to 3, even if the effects are large). The things you propose comparing with each other and the sequence of tests you propose are also dependent.

With appropriate assumptions, even at very small sample sizes you can make some such comparisons, but the power will be very low, and your dependence on the particular assumptions may be the major factor in your conclusions.

• Hi,I'll try to explain more about our data: we analyze 5 times in a row the sames gas cylinder to estimate the concentration of a given gas in the bottle.We want to study if we should take into consideration all 5 values to give an estimate of the concentration or if we should 62.970, 62.202, 61.791, 56.588, 59.155 – alco Apr 7 '13 at 18:48
• Hi,I'll try to explain more about our data: with a gas chromatograph we analyze 5 times in a row a gas cylinder to estimate the concentration of a given gas in the cylinder. We want to study if we should take into consideration all 5 values to give an estimate of the concentration or, on the contrary, if we should not consider the 1st, or the 1st and the 2nd values, when estimating the concentration because these first values are affected, for example, by "unclean"/"non-stabilized" system. An example of the values we obtain are: 62.970 ppb, 62.202 ppb, 61.791 ppb, 56.588 ppb, 59.155 ppb.Help – alco Apr 7 '13 at 18:59
• One possibility is to consider some sort of decaying effect that rapidly goes to zero, in order to estimate where the effect ends rather than guess that it ends at 1 or 2. – Glen_b Apr 15 '13 at 5:53

You can consider some kind of jackknife approach in which each reading is compared against the mean of all other runs except the reading being contrasted.

For example, in one of your comments you mentioned a set of number:

$62.970, 62.202, 61.791, 56.588, 59.155$

To contrast the first one with the rest:

1. Calculate the means of $62.202, 61.791, 56.588, 59.155$, which is $59.934$.
2. Find out the percent difference: $(62.970 - 59.934)/59.934 \times 100\%$, which is $5.07\%$
3. Repeat this for the rest, the whole data should be: $5.07\%, 3.45\%, 2.59\%, -8.03\%, -2.85\%$

Now, this tells a lot of problems. First your assumption that readings get more consistent is challenged here. The "odd-one-out" is actually the fourth run. Another worrying sign is the consistent autocorrelation. A consistent test shouldn't have all the positive error in the front, and negative error at the back, aka, the positive and negative should scatter.

This brings back to your original question. I'd suggest that statistics isn't the core problem here, but the quality control of your device. If "unclean" or "non-stablized" is your concern, then please check the system manual and investigate how to properly rinse your coil, and how to use test run reagent to estimate the efficiency. If you come back to judge your readings in such a post-hoc manner, "weeding" out weird data, your results can invite substantial suspicion.

In a nut shell, I'm not exactly a lab-technician, but from a general researcher's point of view all these data cleaning criteria should be set a prior, that is before you have gotten any data. Once you have the data, and the protocol was followed, and there wasn't detected problem, then the number should be analyzed.

Given a data set $\{y_1, y_2, \ldots, y_n\}$ with $n$ entries, the mean $\mu$ satisfies $n\mu = \sum_{i=1}^n y_i,$ while the mean of the expurgated set $\{y_1, y_2, \ldots, y_{n-1}\}$ is $$\hat{\mu} = \frac{1}{n-1}\sum_{i=1}^{n-1} y_i = \left.\left.\frac{1}{n-1}\right(n\mu - y_n\right)$$ which will equal $\mu$ exactly if and only if the deleted entry $y_n$ equals $\mu$. Thus, deleting an entry from a data set will change the mean unless the point deleted happens to equal the mean of the original data set.

The variance $\sigma^2$ of the original data set satisfies $(n-1)\sigma^2 = \sum_{i=1}^{n}(y_i-\mu)^2$. If we delete an entry (say $y_n$) which happens to have value $\mu$ (so that the mean remains the same), then the expurgated data set has variance \begin{align} \hat{\sigma}^2 &= \frac{1}{n-2}\sum_{i=1}^{n-1} (y_i-\mu)^2\\ &= \frac{1}{n-2}\sum_{i=1}^{n} (y_i-\mu)^2 &\text{since}~ y_n-\mu = 0,\\ &= \frac{n-1}{n-2}\sigma^2\\ &> \sigma^2. \end{align} Thus, our effort to preserve the mean necessarily increases the variance.