2
$\begingroup$

This question follows my previous one. I will now provide my data in order to help you help me :)

As previously said, I have N=42 from a controlled experiment. People made a questionnaire assessing their mood and then played a videogame. They obtained a score from the game.

According to the questionnaire score, they may either be considered as "negative", "neutral" or "positive", respectively represented by the colors red, grey and green.

The following is the plot of the untouched data.

Game Score & Mood

I would like to do some hypothesis testing on score means, like Mean(positive people) > Mean(neutral people).

As I learned from the previous question, if the data is approximately Normal, I can use the T-test. Because histogram does not show interesting facts, I bootstrapped the score mean.

What follow are the results of the bootstrap

Bootstrap for the mean

Call:
boot(data = Tol$Tol.Score, statistic = mean.fun, R = 1500, sim = "ordinary")


Bootstrap Statistics :
    original   bias    std. error
t1* 2817.702 4.079778    91.06932

How much am I safe to perform the tests using the T-test? Shall I use a permutation test instead? Other thoughts?

Extra question: there is one subject that I consider as an outlier. He/she obtained a score < 5. What should I do with it? From what I learned, me shall never remove the outliers. But, since N is 42, it particularly affects the mean of its class (i.e., neutral)

Thank you again for your help.

Context: experiment that will be part of my MSc thesis and maybe a scientific paper.

Update: the dotted colored lines are the mean of scores the three groups

|cite|improve this question
$\endgroup$
  • $\begingroup$ Why are you grouping instead of, e.g., running a regression? $\endgroup$ – jbowman Feb 12 '12 at 15:09
  • $\begingroup$ They are grouped because of previously defined parameters. That is, those with mood < k are negative, those between k and j are neutral and those > j are positive). Honestly, I am not really prepared for the "regression" subject but, from what I studied, I can still use t-tests or permutation tests for mean comparisons. Keep in mind that I really am a newbie with Statistics :) $\endgroup$ – dgraziotin Feb 12 '12 at 15:14
  • 4
    $\begingroup$ On a not very related point, you should know that ~10% of men are red-green colorblind. Many people will not be able to read your figure at all. If you were to send your paper out for review w/ 3 white male reviewers, there is a 1/4 chance that >=1 couldn't read it. I would strongly advise you to use different colors. Some additional info can be found here: stats.stackexchange.com/questions/16631/… Best of luck with your project. $\endgroup$ – gung - Reinstate Monica♦ Feb 12 '12 at 16:49
  • $\begingroup$ Cheers @gung for your advice, I will change the colors of course! $\endgroup$ – dgraziotin Feb 12 '12 at 16:54
3
$\begingroup$

First, with respect to the outlier, I personally would remove it (with the permission of my advisor, of course) - a score of roughly 0 indicates that they couldn't figure out how to play the game, or they just plain hated it, or some other non mood-related factor which completely dominated everything else for that observation.

Second, since you're comparing three means rather than two, or so it seems from your description, you're really looking at an F-test rather than a t-test. A nonparametric alternative to the F test (which assumes Normality as does the t-test), is the Kruskal-Wallis test. It's quite efficient even at the Normal distribution. You might prefer the Jonckheere-Terpstra test, though, which tests against ordered alternatives and therefore would be more powerful given your actual alternative hypothesis.

However, it must be said that the t-test is quite robust with even a smallish sample size, except that outlier of yours would cause problems, no doubt about that.

A permutation test against ordered alternatives would require some care in constructing the test statistic. I'm not so up on my permutation tests as I ought to be, but I'm sure it's been done many times before. Permutation tests are good, no question about it.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thank you, your answer came as I was reading about the Kruskal-Wallis test. Yes, I have three means but I always compare them in two, and always have H0: mean_a = mean_b vs. Ha: mean_a > mean_b. Does your answer still hold? $\endgroup$ – dgraziotin Feb 12 '12 at 15:45
  • $\begingroup$ By comparing two means I do not cover all combinations, that (I think) would imply to compare all three means $\endgroup$ – dgraziotin Feb 12 '12 at 15:56
  • 2
    $\begingroup$ You can certainly do that, but you need to think about the en.wikipedia.org/wiki/Multiple_comparisons problem. Also, your test results won't be independent of each other, e.g., if A > B and B > C, what are the odds that A > C (statistically significantly speaking.) In the pairwise comparison approach, I'd use en.wikipedia.org/wiki/Wilcoxon_signed-rank_test, and I'd still toss the outlier, but permutation would be good too. $\endgroup$ – jbowman Feb 12 '12 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.