I did an experiment with 12 participants. They were asked to rate 3 stimuli, with the following 3 possible answers: "increased", "decreased", "no difference". I have as a result this table:

          Increase Decreased No_difference 

StimulusA   10      1           1
StimulusB   12      0           0
StimulusC    8      2           2

Now,if I want to understand in one shot if the "Increase" answer is chosen significantly more than the other two answers in all the Stimuli, is it correct if I simply use the binomial distribution summing the successes along all the 3 stimuli?

I mean, is it correct doing this?

> prop.test(30,36)

    1-sample proportions test with continuity correction

data:  30 out of 36, null probability 0.5 
X-squared = 14.6944, df = 1, p-value = 0.0001264
alternative hypothesis: true p is not equal to 0.5 
95 percent confidence interval:
 0.6652978 0.9303666 
sample estimates:

Or it is better to use the function in this way instead?

> prop.test(c(10,12,8),c(12,12,12),c(0.5,0.5,0.5))

    3-sample test for given proportions without continuity correction

data:  c(10, 12, 8) out of c(12, 12, 12), null probabilities c(0.5, 0.5, 0.5) 
X-squared = 18.6667, df = 3, p-value = 0.0003204
alternative hypothesis: two.sided 
null values:
prop 1 prop 2 prop 3 
   0.5    0.5    0.5 
sample estimates:
   prop 1    prop 2    prop 3 
0.8333333 1.0000000 0.6666667 

If what you mean to test is whether more people reported an increase than the combined number who reported a decrease or no difference (which is what I think you mean) then your first version is closer to the correct one. Your null hypothesis in that case is that people choose 50-50 between "increase" and "no increase, or decrease" and you are open to evidence either way (greater or less than 50% choose increase).

However, you actually are interested in testing the alternative hypothesis that >50% chose it, so you need a one sided test. You can call this explicitly in prop.test by stating that your alternative hypothesis is only for p being greater than 0.5:

prop.test(30,36, p=0.5, "greater")

It's worth pointing out though that there is nothing special about the 0.5 proportion here - why have you chosen it as the cut-off point for your alternative hypothesis? For example, why not have as a null hypothesis that a third of people choose each option? or any other set of probabilities, perhaps based on an experiment of having people fill in the survey having received a placebo. Having said that, there is strong intuitive appeal in the 0.5 cut-off point and there is no doubt your experiment shows statistically significant evidence that more than 50% do report an increase. The only question is, how does this compare to the percentage who report an increase under other circumstances? (which is not what you've asked here, so I won't worry about it).

  • $\begingroup$ Great answer and even better questions. $\endgroup$ – Roman Luštrik Jan 23 '12 at 11:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.