First, I am not great with stats, it was definitely my weakest class in undergrad/honours, so please be patient (and gentle!)

I have a data set with responses to a 4 point survey item, gathered from 12 groups (which differ in size). My goal is to determine whether the groups are sufficiently similar so as to suggest that they belong to the same population (i.e. are a representative sample).

My first step was to run an ANOVA, which tells me that the means are significantly different between groups, but I'd like to see how their distributions compare too before ruling out the hypothesis that the groups' responses are similar between groups, not just the mean. I figured that although the means may vary, if the distributions of each group overlapped sufficiently, I could argue that they were similar.

I've looked at the coefficient of variation, which "[normalizes] the standard deviation so that it can be compared across various mean scales", I got the SD of the means of each group, CV = .47, which, it is suggested, indicates relatively low variation between group means.

I've also looked at KS test (which I am very vague on), but that appears to be suited only to 2 groups?

I'd love some guidance (handholding). I've always been afraid of stats since I first struggled, and now really regret not learning more.

  • $\begingroup$ I don't think the CV will be any help (nor would the standard deviation) -- the distributions might not overlap at all yet they could have the same standard deviation, or they could have the same CV. (Alternatively they could overlap substantially yet have different sd or different CV.) $\endgroup$ – Glen_b Oct 13 '15 at 10:42

If the means are different, the distributions are different, regardless of CV. What you want to look at is the magnitude of the differences in the means. You might argue, for example, that while these differences are statistically significant, they aren't practically significant.

  • $\begingroup$ This is more a comment than an answer. Please elaborate, by adding some example. $\endgroup$ – user81847 Oct 5 '15 at 6:22
  • $\begingroup$ I'm not sure I can make up a reasonable example without knowing more about the purpose of the experiment $\endgroup$ – bramtayl Oct 5 '15 at 6:41
  • $\begingroup$ Thanks for your input. In this case I am looking at the mean response for a particular survey item (1-4 scale), across 12 groups. I am trying to determine if that overall group of 12 groups can be seen as representative i.e. sufficiently similar so as to use that data as a benchmark for comparing new groups $\endgroup$ – Alex Oct 5 '15 at 21:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy