1
$\begingroup$

Suppose I have the average test results for 7 topics (A, B, C, D, E, F) as percentages (avg performance on A is 90%, B is 80%, etc...). I'm looking to analyze these results in a number of different ways, and although I've taken a number of statistics courses, I want to be careful applying what I've learned to a new problem.

The main questions I'm looking to have answered are:

  1. Is there a statistically significant variation between groups? (and thus I'd use a one-way ANOVA test)
  2. If so, which group(s) is/are significantly different? (this one is a bit harder... not sure how I would analyze this)
  3. I'd like to plot CIs for each group, how should I begin to construct these?

Overall, what other methods are at my disposal/am I forgetting?

$\endgroup$
  • $\begingroup$ When you say groups, does that reflect topics A to F? Or do you have 7 topics each of which has been taken by different groups? $\endgroup$ – David Ernst Dec 9 '16 at 6:33
  • $\begingroup$ Yes, sorry when I mentioned "groups" I meant the "groups of topics". $\endgroup$ – sir_thursday Dec 9 '16 at 14:22
1
$\begingroup$

You're heading in the right direction.

  1. It sounds like an ANOVA is the right way to go, given that your proportions are sample means. If you don't find a significant difference between groups, stop. Otherwise, you can move to step two.

  2. Perform multiple pairwise comparisons (independent two sample t-test) using a Bonferroni correction (adjusted p-value). If the variance and sample sizes are different for each topic, you could look into Welch's t-test.

  3. You can use the standard methodology to compute your CIs. For each topic, identify the sample mean, compute the standard deviation, and then plug it. To see the basic approach, check out: https://en.wikipedia.org/wiki/Confidence_interval#Basic_steps

Note that it's worth considering whether your sample means fulfill the assumptions for an ANOVA. It sounds like your means are drawn from independent samples, but I might be concerned with the normality assumption. That said, if the distributions are unimodal and generally symmetrical, you may be okay. See more about the assumptions here: https://en.wikipedia.org/wiki/Analysis_of_variance#Assumptions_of_ANOVA

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.