# How to quantify variation within one experiment?

Given a couple of measurements I get by running an experiment, I need to express the fact that they are similar to some extent. So, I computed their coefficient of variation, i.e., stdev/mean.

Here is a sample:

3.76025 2.98375 3.78946 3.72195 3.11426 3.60366


and CV = 10.12%

Now I am confused, I don't know how to interpret that number, obviously the lower the better, but what is low? E.g., is 20% and below considered low?

Here is some more of my results: http://pastebin.com/raw.php?i=sg5S05rM
I have thousands of such lines, and all I want to say is that numbers within each line are always more or less similar.

Should I use another measure?

• What do you mean when you say "similar to some extent"? Do you mean the have similar means and standard deviations? Do you mean they are correlated? Do you mean something else? The statistical approach should reflect your goal. Can you say a little more about your goal? Aug 3, 2012 at 12:40
• @JoelW. Please have a look at my updated question. thanks.
– DED
Aug 3, 2012 at 14:54
• Thanks for sharing the lines of data. Still, what do you mean by "more or less similar"? Do you mean have the same pattern (e.g., high score for the third column) or similar means or what? If you clarify what you mean by similar, it will be possible to answer your question. Aug 3, 2012 at 16:40

In my experience coefficient of variation is particularly used for variables with positive means. I said "intended" not defined. A CV makes no sense for zero mean random variables. I see it having value especially for random variable that are strictly positive. I could have a similar value for strictly negative variables but I would want it to be the standard deviation divided by the absolute value of the mean. For variable with very small means a very high coefficient of variation does not seem to me to tell you anything. But a high coefficient of variation relative to a large mean does in my view.

So I think determining what is a good low CV really depends on the actual mean. Two variables with 0 mean, one with a small variance and the other with a large one will both have an infinite coefficient of variation and the CV cannot be used to determine similarity.

Also what to call small is inherently subjective. It is not a statistical issue.

Based on the rows of data you provided I have two comments. (1) Although there is not a great deal of data there certainly appears to be similarity across each row. The particular row that had the 10.12% CV seemed to my eye to have the greatest variation of any of the rows. Now the within row variability seems small compared to the variability between rows which is large. Are these differences due to selection of unrelated data or is there some commanality between rows? (2) In the later case a two-way analysis of variance might show a large and statistically significant effect between rows but an insignificant effect across rows. If there is no relationship then maybe each row can be tested for similarity using one-way ANOVAs. Numbers are just number. To give good statistical advice we need to know the meanings of the numbers.

• Please have a look at my results pastebin.com/raw.php?i=sg5S05rM
– DED
Aug 3, 2012 at 14:50
• All I want to say is that numbers within each line are always more or less similar.
– DED
Aug 3, 2012 at 14:51

This depends on what you think the scale variance of your number are. If you think that the absolute variation about the mean should depend on the mean (e.g. $10 \pm 1$ is about the same as $100 \pm 10$), then CV is a fine choice because it will control for this. If, however, you think that the amount of variation shouldn't depend on the mean ($10 \pm 1$ is about the same as 100 $\pm$ 1), then dividing by the mean is not desirable, and you should use something like variance or standard deviation ($\sigma$). A 95% confidence interval is given by $\mu \pm 1.96 * \sigma$. You could even use the range (max - min), though that will be more susceptible to outliers.

As far as whether a particular statistic is "big" or "small", that depends completely on what you're measuring and how you feel about it. Standard deviation will be in the same units as your measurements, CV will be a proportion (percent if you divide by 100). You say your data is for time elapse. A 20% CV could be fine for something where you expect wide variation, maybe how long a whale stays underwater, but it sounds very high for something that should be regular. Similarly, a standard deviation of 1 minute for how long it takes me to walk from my office to another building seems reasonable, about the same. If it is 1 hour, I'm less happy about the situation.

• Actually, as you can see in my data file, I made the observation that each time I run my experiment, my six values are ~ about the same. "Same" for me means 0.2~0.4~0.56 as 10~14~8. May be the same range or order of magnitude. May be I should compare CVs across lines after all. In fact all I need is to express my observation statistically :)
– DED
Aug 4, 2012 at 9:49
• Based on that, I agree, use CV. Aug 4, 2012 at 19:48