I have conducted a survey in which the participants were asked to answer questions using 3 different techniques. At the end they were asked which technique the preferred, the results are as follows

  • A = 26
  • B = 8
  • C = 6

I want to verify that technique A is better than the other techniques. I was thinking of using the chi-squared test, with the null hypothesis that all media are equally good. So my expected value would be 13.3. I want to know if this is the right way to do it?

  • $\begingroup$ Did you hypothesize that A is preferred a-priori, or are you wondering that now because of the strong preference in the data? $\endgroup$ – gung - Reinstate Monica Mar 22 '16 at 22:17
  • $\begingroup$ I did hypothesise (hope) that there will be strong preference for A. $\endgroup$ – salman Mar 22 '16 at 22:26
  • $\begingroup$ @gung Can you please expand on your comment? $\endgroup$ – Antoni Parellada Mar 24 '16 at 10:31
  1. We want to compare different counts or proportions to a theoretical uniform distribution. Hence, we are setting up a goodness-of-fit test.

  2. This can be calculated as an exact procedure using a multinomial test, or with an asymptotic approximation, using a chi-square goodness-of-fit. Although both tests yield a significant result, the difference between the exact multinomial test and the chi square is quite remarkable.

In R, the two calls are:

observed <- c(A = 26, B = 8, C = 6); prob <- rep(1/length(observed), 3)

For the multinomial test {EMT}:

multinomial.test(observed, prob, useChisq = FALSE, MonteCarlo = FALSE)
 Exact Multinomial Test, distance measure: p

    Events    pObs    p.value
       861       0     0.0002


chisq.test(observed, p = prob, correct = F)
    Chi-squared test for given probabilities
data:  observed
X-squared = 18.2, df = 2, p-value = 0.0001117

These are "omnibus" tests, and post-hoc comparisons are needed. In this regard, this post was useful, and already indirectly discussed by @Glen_b here. The idea is to compare each one of the proportions to the aggregate of the other two groups with a binomial test, and compensate with the Bonferroni correction.

In this case, since we are doing $3$ comparisons, and with a significance level of $0.05$ the correction is: $0.05/3 = 0.01666667$.

But we really won't need it, because the results are quite obvious after running the first test, which doesn't even come close to being questionable:

binom.test(26, sum(observed), 1/3, alternative="greater")

    Exact binomial test

data:  26 and sum(observed)
number of successes = 26, number of trials = 40, p-value =
alternative hypothesis: true probability of success is greater than 0.3333333
95 percent confidence interval:
 0.5080545 1.0000000
sample estimates:
probability of success 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.