1
$\begingroup$

I am quite new to Statistics, so please do not flame me for such a simple question.

I made a controlled experiment and successfully gathered the data. 41 persons participated. They made a questionnaire and a game. From both of them, they obtained a single, discrete score.

The situation is the following:

I split the 41 people in two groups A and B (those that obtained a score <= 0 in the questionnaire vs. those that obtained a positive score in the questionnaire). Actually, A is composed by just 6 of them but different splits may be possible under different reasoning. Let's just keep this one for now.

Therefore, I have two means for the game, X and Y, respectively. I would like to see if there are significant evidences that Y > X.

Which test should I use? I am really confused.

I continue to read around that if the sample size is > 30, I may use the T test anyway.

Then, there is the Mann–Whitney U test to be used for two samples if we can not assume Normal Distribution but should be applied for two samples of independent observation.

Any hints? Thank you very much.

Update: Ideally speaking, the population could also be categorized by the questionnaire score, I am trying to see if there is statistical evidence that people with a positive score also would have a higher game score

Improve this question
$\endgroup$
3
  • $\begingroup$ Simple questions are just fine here, thanks. (That doesn't mean that they will get simple answers, though. :-) $\endgroup$ – whuber♦ Feb 9 '12 at 22:06
  • $\begingroup$ @whuber Thank you..I will try to start a PhD in November, therefore I need to learn a lot about statistical analysis :-) $\endgroup$ – user8438 Feb 9 '12 at 22:13
  • $\begingroup$ @whuber thank you also for the better question title. It also helps me to clarify some terms $\endgroup$ – user8438 Feb 9 '12 at 22:31
1
$\begingroup$

To resume I think your question is whether you can reject the null hypothesis that the B group has a significantly higher average game score than the control group A.

For this you would have to first calculate a test statistic for your experiment, then test for significance. The test statistic can be defined as $$\frac{\hat{Y} - \bar{Y}}{SE}$$ where $\hat{Y}$ is the observed group B average score, $\bar{Y}$ is the group A expected average score and $SE$ is the standard error of the group B scores, or $\sqrt{N} * SD$.

Now to test for significance when the number of observations is small we want to use the Student's curve, rather than the normal curve. It complicates the P-value calculation a bit, because you will have to choose the curve that match your group degrees of freedom (df = observations - 1), but each one of these specific curves give a better probability distribution for reduced sample sizes. This final number will give you the probability that we can't reject the null hypothesis. Standard practices are that if the p-value is < 5% we can reject the null hypothesis but in practice I would be less strict, especially for small sample sizes. I would say a p-value < 10% is worth investigating.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Therefore, can I always use the Student's T-Test in such cases? If I can not assume Normal Distribution, always use T-Test when comparing means, right? $\endgroup$ – user8438 Feb 10 '12 at 12:23
  • $\begingroup$ Yes. The T-test is valid in any case. What changes in this case with small samples is the distribution curve, that is normal vs. Student. Also you might want to calculate the standard deviation using the unbiased estimator N-1: en.wikipedia.org/wiki/Unbiased_estimation_of_standard_deviation $\endgroup$ – Robert Kubrick Feb 10 '12 at 12:56
  • $\begingroup$ Thanks, this is everything I needed. One last question just to be sure: if there were 1000 people participating, because I can not assume normal distribution of the population, still the T-test would have been the right one? $\endgroup$ – user8438 Feb 10 '12 at 13:14
  • $\begingroup$ I am saying this because the first lines in the Wikipedia page about Student's T distribution say "Student’s t-distribution (or simply the t-distribution) is a continuous probability distribution that arises when estimating the mean of a normally distributed population in situations where the sample size is small and population standard deviation is unknown". In this case, I can not tell that the population is normally distributed $\endgroup$ – user8438 Feb 10 '12 at 14:21
  • $\begingroup$ For 1000 observation I would definitely switch to a normal curve and use the regular SD formula. I would check the distribution histogram to make sure it is normal, but 1000 is a relative good sample number. Generally the student dist tables go up to around 100 df, but I can't give you an exact threshold. $\endgroup$ – Robert Kubrick Feb 10 '12 at 14:55
0
$\begingroup$

I am trying to see if there is statistical evidence that people with a positive score also would have a higher game score

Looks to me that you need some correlation coefficient. I suspect you assign subjects a discrete score from a fixed interval, say from 0 to 50 or so. In that case I recommend you to compute Spearman's rank correlation coefficient.

It will measure the strength of the game-score vs. questionnaire-score relation. A correlation coefficient goes from -1 (there is a perfect negative relation) to +1 (perfect positive relation. In your case, you want it to be as close to 1 as possible, meaning that high scores in the questionnaire correspond to high scores in the game.

You can compute it with R for example, by using cor(questionnaire, game, method="s").

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy