SD larger than mean, non-negative scale

Question

I was given an article reporting a study very similar to one my lab wishes to run. But, I noticed that for the variable of interest, Duration, the SDs are larger than the mean...since this is duration measured in minutes it can never be negative and this seems very strange to me. This happened in 2 studies reported, below is one.

Beyond that, this is a mixed-design. Control v Treatment (between groups), and Time1,Time2,Time3 (repeat measures). Here are the means (SDs), N >200

                       Time1                Time2                  Time3 
Control               15.1 (14.6)          14.4 (14.8)            13.3 (15.7)
Treatment             14.8 (13.2)          10.0 (12.2)            8.2 (9.9)

...they ran an ANOVA and reported a p<.001.

I was asked to use this as a basis for a power analysis to determine sample size for our study. I am pretty sure this indicates that the data is non-normal or has outliers and I don't feel comfortable determining sample size based on this. Am I just way off base?

Answer 1

It's easily possible for the standard deviation to exceed the mean with non-negative or strictly positive data

I'd describe the case for your data as the standard deviation being close to the mean (not every value is larger and the ones that are larger are generally close). For non-negative data, it does pretty clearly indicate that the data are skew (for example, the gamma distribution with coefficient of variation =1 would be the exponential distribution, so if the data were gamma, they'd look somewhere near exponential)

However, with that sort of sample size, the ANOVA may not be particularly badly affected by that; the uncertainty in the estimate of pooled variance will be pretty small, so we might consider that between the CLT (for the means) and Slutsky's theorem (for the variance estimate on the denominator), an ANOVA will probably work reasonably well, since you'll have an asymptotic chi-square, for which the ANOVA-F with its large denominator-degrees-of-freedom will be a good approximation. (i.e. it should have reasonable level-robustness, and since the means are not so very far from constant, the power shouldn't be too badly impacted by the heteroskedasticity)

That said, if your study will have a smaller sample size, you may be better off looking at using a different test (perhaps a permutation test, or one more suitable for skewed data perhaps one based on a GLM). The change in test may require a somewhat larger sample size than you'd get for a straight ANOVA.

With the original data you could do a power analysis under a suitable model/analysis. Even in the absence of the original data, one could make more plausible assumptions about the distribution (perhaps a variety of them) and investigate the entire power curve (or, more simply, just the type I error rate and the power at whatever effect size is of interest). A variety of reasonable assumptions could be used, which gives some idea of what power may be achieved under plausible circumstances, and how much larger the sample size might need to be.

Answer 2

You are correct in concluding that the data is non-normal. If the data were normal then we would expect about 16% of observations to be less than the mean minus the standard deviation. With an SD larger than the mean this number is negative and you state that there cannot be negative numbers, so what you are seeing is not consistent with normally distributed data. The SD values are possible, but only if the distribution is very right skewed (which is common in durations).

I agree that choosing a sample size based on assuming data will be normal is not a good idea, but if you can find out more about the process and find a right skewed distribution (a gamma distribution as one possibility) that is a reasonable assumption, then you could use that to help determine sample size.