I'm struggling with analysing the following categorical data shown in the barplot. There are 5 categories (R, D, E, N, A) and I have the number of votes (number of respondents) for each one. As shown in the figure, the number of respondents choosing category R (21) and D (20) are very similar. I was wondering if there is a statistical test I can use to test which category should be chosen? Essentially, I want to know how big the difference in number of respondents need to be between the two most popular categories in order to be able to choose one of them with statistical significance.

I wanted to use Fisher's exact test for counts by studying the ratio 21/55 and 20/55, 55 being the total number of respondents. However, all the examples I have come across are analysing 2x2 contingency tables. Whereas in this case there's only 1x2. Alternatively, I was wondering if all five categories can be used in the test somehow. However, I still don't have the two usual groups such as man/ woman or medicine/ no medicine. Any ideas on how I can still use Fisher's test?

I've also already done a Chi-square test with expected frequencies equal to (1/5, 1/5, 1/5, 1/5, 1/5) which returned a significant p-value (rejecting H0). Any ideas on further analysis? enter image description here

  • $\begingroup$ Just added response to "[H]ow big [does] the difference in number of respondents need to be between the two most popular categories in order to be able to choose one of them with statistical significance[?]" to my Answer. $\endgroup$
    – BruceET
    Apr 21 '20 at 9:54

You already know from a chi-squared test that the distribution is not uniform across all five categories. You seem to want to look more deeply than that.

Looking at the top two categories. If you're doing an ad hoc test, just looking at Categories R and D, asking whether Fisher's exact test finds a significant difference between 21 in 41 and 20 in 41, the answer is No. It would be impossible to get a more equal split between R and D. In R statistical software, we get P-value $1 >> 0.05:$

DTA1 = rbind(c(21,20), c(41,41))
     [,1] [,2]
[1,]   21   20
[2,]   41   41


        Fisher's Exact Test for Count Data

data:  DTA1
p-value = 1
alternative hypothesis: 
  true odds ratio is not equal to 1
95 percent confidence interval:
  0.4636901 2.3809126
sample estimates:
odds ratio 

You wondered how big the difference between the top two categories would need to be in order to reach statistical significance. In this instance, the least extreme split that is significant at the 5% level would be 29 in 41 vs. 12 in 41. (The split 28 in 41 vs. 13 in 41 has P-value $0.0577.)$

fisher.test(rbind(c(29,12), c(41,41)))$p.val
[1] 0.03434586

Combining categories. If you're asking whether R & D together have a significantly greater count (41) than E, N & A together (14), according to Fisher's Exact Test, then the answer is Yes.

DTA2 = rbind(c(41,14),c(55,55))
[1] 0.002694403

Anyhow, these examples show you how Fisher's test for a $2 \times 2$ table works in R.

If you're going to do several such ad hoc tests, then you need to use Bonferroni (or some other method) to guard against false discovery.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.