1
$\begingroup$

This might be a simple question but I am unable to find a solution on the web. I would like to measure variation of some numerical data. What is the minimum number of values I should use for measuring their standard deviation or coefficient of variation? Basically my plan is to derive groups from a set of performance metric values (all are positive integers). I would like to choose "variation" measure for grouping. My idea is as follows: For example, I have 16 positive integer numbers. First I would like to sort them then divide them as four groups. Each group contains 4 integers and I will calculate variation in terms of either standard deviation or coefficient of variation. If the difference in the variation of two groups is less than a threshold then I will merge them, and etc. Here my question is, can I take a data size of 4 for calculating variation? Please help me.

$\endgroup$
  • 1
    $\begingroup$ Your approach is very non standard and that is probably why you don't see the approach or an answer to your question on the internet. If it is all from one population and you want to estimate the population variance or standard deviation you should just use the sample estimate of in that you can find in any elementary statistics book. There is no relevance to the fact that your data are noninteger. The accuracy of the estiamte depends on the sample size and the population variance or standard deviation. There is no one sample size that applies to every problem. But it looks like you might be c $\endgroup$ – Michael R. Chernick May 23 '12 at 19:58
  • $\begingroup$ Oops.. my bad. I wanted to use the words "positive integers". Somehow I produced a typo. Yes, I am trying to clustering data. Can I use a sample size of 4 for calculating variation? Please let me know. $\endgroup$ – samarasa May 23 '12 at 20:07
  • $\begingroup$ I think I will drop out of this one. There are many ways to cluster data. I am not sure that what you are proposing falls into a known approach that has been well researched. The question is too vague right now for an answer. Ther should be some desirable properties that you want your algorithm to have . Acheiving it requires that it can be specified and that the result is a direct function of the sample size. $\endgroup$ – Michael R. Chernick May 23 '12 at 20:23
  • $\begingroup$ One last and quick question. Is it wrong to use a sample size of 4 or 5 for calculating variation? $\endgroup$ – samarasa May 23 '12 at 20:29
2
$\begingroup$

In the book Statistical Rules of Thumb (explained by Dan Goldstein here), Gerald van Belle describes how the width of a Student's t confidence interval decreases with more observations. Here is the useful chart:

confidence interval asymptote

His "rule of thumb" is to gather at least 12 data points for a given sample. But as you can see from the chart, this is not strictly necessary. I don't fully understand your question, but the answer is that you can get a measure of the variation from 4 data points. In fact, you could get one from 3.

However, the more data you include in each group, the more accurate your procedure will be. So, if you have the option, you might consider gathering enough data to get 6 observations per group, or splitting the data into only 3 groups instead of 4 (with the odd 6th datum assigned randomly to one of the them).

$\endgroup$
2
$\begingroup$

There is nothing wrong with getting estimates based on random samples if small sizes. However the estimates have larger variances than for larger sample sizes. So a decision on sample size depends on how small you want that variance to be.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.