1
$\begingroup$

I have this problem and more I read online, more confused I'm getting. So I have 4 (or more) groups with different users (for simplicity I only used small numbers, but I may have 50, 100, 150 and 200 users per group). After finding mean for each group, I want to be confident and back the results with some stats, but not sure what. Looking for a statistical metric telling me when I can disregard mean e.g. from Class1 due to not enough observations. I have measure SD, Variance, SE and Confidence Intervals (CI), but not one value to show relation to sample size (although CI does that partially). Appreciate any input.

enter image description here

$\endgroup$
1
$\begingroup$

An ANOVA would be my first choice. With an ANOVA you perform a global test of difference for all the groups. If you reject this hypothesis (of at least one difference in means) you can perform additional tests to test each pair of means, for ex. post-hoc Tukey HSD test. Both of these tests make no assumptions of group sizes.

$\endgroup$
  • $\begingroup$ Thanks for suggestion user2974951. I thought of that, but that doesn't give one value per group, but in contrary does for all the groups (as you have mentioned). And HSD test is not feasible, especially if one have a lot of groups to compare. What I think is closed to a solution for my issue may be some power analysis. $\endgroup$ – Altin Feb 6 at 15:42
  • $\begingroup$ @Altin I disagree, how many groups do you have? Also power analysis is completely different and not all related to comparing means. $\endgroup$ – user2974951 Feb 6 at 15:43
  • $\begingroup$ Hi, maybe I formulated my question little incorrectly. What I want to do is to stand by my calculations. A better question will be: When I can say with confidence that my calculation reflect correct mean for the sample population. I'm not too confident with the mean from 3 users, but i'm more if I had 30 / 50 / 100 users. And I want to represent that WHEN by a statistical value. $\endgroup$ – Altin Feb 6 at 17:54
  • $\begingroup$ @Altin That does sound like power analysis. You should edit your question and include this new information. Make sure you state your question clearly, also change your title name, since it does not match. $\endgroup$ – user2974951 Feb 7 at 7:42
0
$\begingroup$

Thanks for comments. Following suggestion from @user2974951, I think I found what I'm looking for. All I need to use is the Power Calculations with the following R formula:

pwr.p.test(h = 0.2, n = nr, sig.level=0.05)

While I have to understand what all parameters mean and read some literature, first preliminary results seem promising.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.