1
$\begingroup$

I'd like to ask if someone could help me with the following problem:

we have measured the same sample 5 times and we would like to check if there are significant differences in mean and stdev values if we use:

  • All 5 datapoints
  • Only the last 4 datapoints
  • Only the last 3 datapoints

We have performed ANOVA analysis but we are not sure about the results because we might not have homocedasticity.

Which tests would you do to investigate this issue?

Thanks in advance for your help.

$\endgroup$
  • $\begingroup$ As @EngrStudent mentions below, 5 data points isn't much. But, could you perhaps describe your data a bit better? $\endgroup$ – Jason Morgan Apr 2 '13 at 17:30
  • $\begingroup$ @JasonMorgan i suspect that with datapoint, he refers to a single time they measured the sample. Not to the number of observations within a single measuring session. $\endgroup$ – mvherweg Aug 1 '13 at 10:55
1
$\begingroup$

Not only don't you have much data (it's very unlikely to be enough to pick up any change even moving from 5 points to 3, even if the effects are large). The things you propose comparing with each other and the sequence of tests you propose are also dependent.

With appropriate assumptions, even at very small sample sizes you can make some such comparisons, but the power will be very low, and your dependence on the particular assumptions may be the major factor in your conclusions.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Hi,I'll try to explain more about our data: we analyze 5 times in a row the sames gas cylinder to estimate the concentration of a given gas in the bottle.We want to study if we should take into consideration all 5 values to give an estimate of the concentration or if we should 62.970, 62.202, 61.791, 56.588, 59.155 $\endgroup$ – alco Apr 7 '13 at 18:48
  • $\begingroup$ Hi,I'll try to explain more about our data: with a gas chromatograph we analyze 5 times in a row a gas cylinder to estimate the concentration of a given gas in the cylinder. We want to study if we should take into consideration all 5 values to give an estimate of the concentration or, on the contrary, if we should not consider the 1st, or the 1st and the 2nd values, when estimating the concentration because these first values are affected, for example, by "unclean"/"non-stabilized" system. An example of the values we obtain are: 62.970 ppb, 62.202 ppb, 61.791 ppb, 56.588 ppb, 59.155 ppb.Help $\endgroup$ – alco Apr 7 '13 at 18:59
  • $\begingroup$ One possibility is to consider some sort of decaying effect that rapidly goes to zero, in order to estimate where the effect ends rather than guess that it ends at 1 or 2. $\endgroup$ – Glen_b Apr 15 '13 at 5:53
1
$\begingroup$

You can consider some kind of jackknife approach in which each reading is compared against the mean of all other runs except the reading being contrasted.

For example, in one of your comments you mentioned a set of number:

$62.970, 62.202, 61.791, 56.588, 59.155$

To contrast the first one with the rest:

  1. Calculate the means of $62.202, 61.791, 56.588, 59.155$, which is $59.934$.
  2. Find out the percent difference: $(62.970 - 59.934)/59.934 \times 100\%$, which is $5.07\%$
  3. Repeat this for the rest, the whole data should be: $5.07\%, 3.45\%, 2.59\%, -8.03\%, -2.85\%$

Now, this tells a lot of problems. First your assumption that readings get more consistent is challenged here. The "odd-one-out" is actually the fourth run. Another worrying sign is the consistent autocorrelation. A consistent test shouldn't have all the positive error in the front, and negative error at the back, aka, the positive and negative should scatter.

This brings back to your original question. I'd suggest that statistics isn't the core problem here, but the quality control of your device. If "unclean" or "non-stablized" is your concern, then please check the system manual and investigate how to properly rinse your coil, and how to use test run reagent to estimate the efficiency. If you come back to judge your readings in such a post-hoc manner, "weeding" out weird data, your results can invite substantial suspicion.

In a nut shell, I'm not exactly a lab-technician, but from a general researcher's point of view all these data cleaning criteria should be set a prior, that is before you have gotten any data. Once you have the data, and the protocol was followed, and there wasn't detected problem, then the number should be analyzed.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Given a data set $\{y_1, y_2, \ldots, y_n\}$ with $n$ entries, the mean $\mu$ satisfies $n\mu = \sum_{i=1}^n y_i, $ while the mean of the expurgated set $\{y_1, y_2, \ldots, y_{n-1}\}$ is $$\hat{\mu} = \frac{1}{n-1}\sum_{i=1}^{n-1} y_i = \left.\left.\frac{1}{n-1}\right(n\mu - y_n\right)$$ which will equal $\mu$ exactly if and only if the deleted entry $y_n$ equals $\mu$. Thus, deleting an entry from a data set will change the mean unless the point deleted happens to equal the mean of the original data set.

The variance $\sigma^2$ of the original data set satisfies $(n-1)\sigma^2 = \sum_{i=1}^{n}(y_i-\mu)^2$. If we delete an entry (say $y_n$) which happens to have value $\mu$ (so that the mean remains the same), then the expurgated data set has variance $$\begin{align} \hat{\sigma}^2 &= \frac{1}{n-2}\sum_{i=1}^{n-1} (y_i-\mu)^2\\ &= \frac{1}{n-2}\sum_{i=1}^{n} (y_i-\mu)^2 &\text{since}~ y_n-\mu = 0,\\ &= \frac{n-1}{n-2}\sigma^2\\ &> \sigma^2. \end{align}$$ Thus, our effort to preserve the mean necessarily increases the variance.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Comments:

  • Five samples is pretty low. I prefer 98, or 300, or tens of thousands.
  • As I am understanding the question, it seems pretty simple to answer using Excel using brute methods.
  • Really the number of required samples is governed by the function that you are trying to measure. Metropolis-Hastings often takes tens of thousands to get "its feet on the ground".

Answer:

You can use bootstrap resampling to investigate the uncertainty in both the mean and the standard deviation for different sample counts. This will use the data you have, not the date you wish you had, and give you an answer that is about as meaningful as possible. So after you find say 300 estimates of the mean with uniform random sampling with replacement (bootstrapping) then you can perform EDA on those values by graphing the CDF and looking at both central tendency and variation.

I do not feel comfortable taking thousands of resamples when I only have 5 actual measurements. Even 300 is a bit high, but it is at an boundary where results get to be clean and still remain informative.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ You write as if samples do not cost anything. In many fields of endeavor it would be impossible to achieve your preferences and utterly ridiculous to suggest taking tens of thousands of samples. Besides, whether they are your preferences, or those of a Nobel Prize winner, or of a random person on the street, ought to make no difference in an objective answer to a statistical question. $\endgroup$ – whuber Apr 2 '13 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.