I am doing a biological experiment to measure a biologically relevant quantity, say concentration of a chemical in my cells. Ideally, I would do 3 or 5 or some number of replicates of my experiment to get an estimate of mean and standard deviation. But due to time limitation, complexity of my experiment and costs involved, I can only do two replicates. Now, I end up with two estimates of concentration. No one questioned me when I took the mean of these two quantities. But people were uncomfortable when I calculated the standard deviation. I could understand their concern but I want to get more insights into why it is ok or not ok to take standard deviation in this case? If it is not ok, what are my options?
$\begingroup$
$\endgroup$
11
-
1$\begingroup$ If your data is Normally distributed, then don't forget to apply the necessary multiplicative factor to the sample standard deviation to get, say, a 95% two-sided confidence interval. That is based on a Student t distribution with 1 degree of freedom, and is a whopping 12.71, in contrast to the value of 1.96 to which fans of the Normal distribution are so accustomed. $\endgroup$– Mark L. StoneAug 16, 2016 at 22:37
-
1$\begingroup$ I think my comment above, from before your update, is getting to the nub of the matter. Try running that by the uncomfortable people. $\endgroup$– Mark L. StoneAug 16, 2016 at 23:17
-
$\begingroup$ So you mean that the estimate of std dev should be multiplied by 12.71 to get the error bars? $\endgroup$– Mr KAug 16, 2016 at 23:43
-
2$\begingroup$ Yes, 12.71, not 1.96. Of course, the error bars will be wide. But that is the penalty you incur for having such a small sample. $\endgroup$– Mark L. StoneAug 16, 2016 at 23:47
-
2$\begingroup$ If you can get up to 3 data points, the multiplicative factor on the sample standard deviation to get a two-sided confidence interval goes down from 12.71 to 4.30. That's a 66% savings. And if you act in the next 15 minutes, I'll give you some additional "savings" in the form of n-1 vs. n not dinging you as badly in the denominator of the formula for sample standard deviation when n = 3 vs. 2. $\endgroup$– Mark L. StoneAug 17, 2016 at 0:03
|
Show 6 more comments
4 Answers
$\begingroup$
$\endgroup$
9
Compilation and expansion of comments:
Let's presume your data is Normally distributed.
If you want to form two-sided error bars (or confidence intervals), say at the 95% level, you will need to base that on the Student t distribution with n-1 degrees of freedom, where n is the number of data points. You propose to have 2 data points, therefore requiring use of Student t with 1 degree of freedom.
95% 2-sided error bars for n = 2 data points require a multiplicative factor of 12.71 on the sample standard deviation, not the familiar factor of 1.96 based on the Normal (Student t with $\infty$ degrees of freedom). The corresponding multiplicative factor for n = 3 data points is 4.30.
The situation gets even more extreme for two-sided 99% error bars (confidence intervals).
As you can see, at either confidence level, there's a big "savings" in the multiplicative factor if you have 3 data points instead of 2. And you don't get dinged as badly by the use of n-1 vs. n in the denominator of sample standard deviation.
n Confidence Level Multiplicative Factor
2 0.95 12.71
3 0.95 4.30
4 0.95 3.18
5 0.95 2.78
infinity 0.95 1.96
2 0.99 63.66
3 0.99 9.92
4 0.99 5.84
5 0.99 4.60
infinity 0.99 2.58
answered Aug 17, 2016 at 0:36
-
1$\begingroup$ "But people were uncomfortable when I calculated the standard deviation. I could understand their concern but I want to get more insights into why it is ok or not ok to take standard deviation in this case? If it is not ok, what are my options?" -- this question still remains. I am in disagreement of using CIs for only two samples. With n = 2, you basically get a meaningless SD and CIs. $\endgroup$– JonAug 17, 2016 at 15:30
-
2$\begingroup$ @Johnnyboycurtis The confidence intervals are wide, as seen from the very large multiplicative factors arising from a t distribution with 1 degree of freedom. Whether they are 'satisfying" is another matter. The typical uncomfortable people were probably thinking in terms of Normal Z for multiplicative factor, not t with 1 degree of freedom. $\endgroup$ Aug 17, 2016 at 15:33
-
$\begingroup$ whether t or Z you're still assuming the data is approx. Normal after some large sample size; this assumption does not hold well with n = 2. $\endgroup$– JonAug 17, 2016 at 15:36
-
$\begingroup$ I think you need to ask yourself, would you be comfortable presenting this to a room of statisticians? $\endgroup$– JonAug 17, 2016 at 15:37
-
2$\begingroup$ Normal assumption is just as valid whether 2 data points or a million. There's no averaging going on here as in Central Limit Theorem. But yes, Normal assumption is crucial, that's why at the beginning of my answer I stated "Let's presume your data is Normally distributed." I would discuss the Normality assumption if presenting this to anyone, whether 2 data points or a gazillion. $\endgroup$ Aug 17, 2016 at 15:39
$\begingroup$
$\endgroup$
2
Setting aside your initial explanation of the time-series context, it might be useful to look at this as a simple case of observing two data points. For any two observed values $x_1 , x_2$ the sample standard deviation is $s = |x_2 - x_1| / \sqrt2$. This statistic is exactly as informative as giving the sample range of the two values (since it is just a scalar multiple of that statistic). There is nothing inherently wrong with using this statistic as information on the standard deviation of the underlying distribution, but obviously there is a great deal of variability to this statistic.
The sampling distribution of the sample standard deviation depends on the underlying distribution for the observable values. In the special case where $X_1, X_2 \sim \text{IID N}(\mu, \sigma^2)$ are normal values you have $S \sim \sigma \cdot \chi_1$ which is a scaled half-normal distribution. Obviously this means that your sample standard deviation is quite a poor estimator of the standard deviation parameter (biased and with high variance), but that is to be expected with so little data.
-
$\begingroup$ Isn't the standard deviation formula you write the population standard deviation, and sample standard deviation would be more appropriate? Using the sample standard deviation, for n=2 the standard deviation is identical to the range/difference of the two data points, and the relative standard deviation is identical to the percent difference. Thus, the standard deviation is certainly meaningful. $\endgroup$ Jun 22, 2022 at 10:13
-
$\begingroup$ I ran through the derivation, you are correct that the sample standard deviation simplifies to $s=|x_2-x_1|/\sqrt{2}$. $\endgroup$ Jun 22, 2022 at 10:25
$\begingroup$
$\endgroup$
If you only have 2 values, just present those 2 values. It doesn't make sense to convert 2 measurements into 2 other quantities (mean and stdev) if your audience is going to argue about the significance of one or the other.
If you want to estimate uncertainty, these other responses are right on, but don't forget to add other potential sources of error (measurement instrument bias errors, resolution, etc.).
$\begingroup$
$\endgroup$
1
" I know that you could compare the two time series by taking Pearson correlation and such" -- this is incorrect. Pearson Correlation assumes observations are independent, but time series data is by nature not independent. You would actually need to use a cross-correlation. Reference: https://web.archive.org/web/20160302024823/https://onlinecourses.science.psu.edu/stat510/node/74
Also, you shouldn't use the typical variance (if you really must calculate a variance); I suggest using something like Mean Absolute Deviance (MAD). You can then create a histogram to summarise the distribution of similarity/dissimilarity.
-
$\begingroup$ thanks for the answer. please look at my updated question. $\endgroup$– Mr KAug 16, 2016 at 22:52