1
$\begingroup$

I'm trying to see if a given item is significantly more selected than the others.

Let's imagine that I have 7 items A,B,C,D,E,F,G, and that think that most people (of a given group) will prefer to choose A. So I'm doing an experiment, in which I'm asking the subject to choose among three of the items. One item will always be A, whereas the twos others will be randomly drawn for the remaining 6 items. The order in which the items are presented is randomized, as to avoid any order effect.

So basically, A has 1/3 to be chosen, whereas the two other items would be first randomly drawn for the remaining 6 (so 1/6) items, and then would have a 1/3 chance to be selected, thus resulting into (1/18) chance to appear and be selected.

Now, let's imagine that I had the following results, which represent the number of time a given item has been chosen:

A B C D E F G
30 2 20 7 7 6 11

Which test should I use to see if the item "A" is statistically more selected than the others? I was thinking of a chi-square test, but I'm not really sure.

Thank you!

$\endgroup$
2
  • $\begingroup$ This seems rigged to prefer A. Why do people have to pick A? $\endgroup$ – Dave May 1 at 13:16
  • $\begingroup$ Well, it boils down to my hypothesis actually, and it's more of a robustness check that I did in my study to verify that people indeed prefer A rather than something else randomly chosen. For instance, for another group of people (another condition in my survey), the hypothesized preferred item would be B (so B will always be there, whereas the two other items would be randomly chosen). $\endgroup$ – Jauhnax May 1 at 13:17
0
$\begingroup$

If by 'amount of time' you mean 'number of times' and if you intend to test whether all items are equally likely to be chosen, then in R you might use prop.test for which the default null hypothesis is that all seven categories are equally likely to be chosen. (Similar to vetting a 7-sided die for fairness). The null hypothesis is strongly rejected with P-value near $0.$

x = c(30, 2, 20, 7, 7, 6, 1)
n = sum(x);  
prop.test(x, rep(n,7))

        7-sample test for equality of proportions 
        without continuity correction

data:  x out of rep(n, 7)
X-squared = 75.817, df = 6, p-value = 2.605e-14
alternative hypothesis: two.sided
sample estimates:
    prop 1     prop 2     prop 3     prop 4     prop 5     prop 6     prop 7 
0.41095890 0.02739726 0.27397260 0.09589041 0.09589041 0.08219178 0.01369863 

This test is equivalent to a chi-squared test with a 'contingency table' as shown below:

TBL = rbind(x, n-x)
TBL
  [,1] [,2] [,3] [,4] [,5] [,6] [,7]
x   30    2   20    7    7    6    1
    43   71   53   66   66   67   72

chisq.test(TBL)

        Pearson's Chi-squared test

data:  TBL
X-squared = 75.817, df = 6, p-value = 2.605e-14

Note: You seem to have a special interest in category A, but (from what you say) it is not clear whether that interest would naturally result in a particular ad hoc test. For example, an ad hoc test that A by itself is as popular as all other categories combined, had P-value $0.031.$ As a main test, one might find that significant, but maybe not as one of several ad hoc tests on the same data.

prop.test(c(30, 43), c(73,73), cor=F)

        2-sample test for equality of proportions 
        without continuity correction

data:  c(30, 43) out of c(73, 73)
X-squared = 4.6301, df = 1, p-value = 0.03142
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.33769717 -0.01846721
sample estimates:
   prop 1    prop 2 
0.4109589 0.5890411 

By contrast the hypothesis that A and B are equally likely to be selected is very strongly rejected according to almost any ad hoc criterion for avoiding false discovery.

prop.test(c(30, 2), c(32,32), cor=F)

        2-sample test for equality of proportions 
        without continuity correction

data:  c(30, 2) out of c(32, 32)
X-squared = 49, df = 1, p-value = 2.56e-12
alternative hypothesis: two.sided
95 percent confidence interval:
 0.7563921 0.9936079
sample estimates:
prop 1 prop 2 
0.9375 0.0625 

At the start of your question the clause "a given item is significantly more selected" can be interpreted in several ways. Do you mean that a pre-chosen item A is more likely than any one of the others? all of the others taken collectively? Or do you mean that not all items are equally likely to be selected (so obviously, some one item must be more likely than some others)? These interpretations are quite different in consequence, and you should try to decide what you mean and to make a clear statement from the start about what you want to test.

$\endgroup$
2
  • $\begingroup$ Thank you for your very detailed answer. I was not really clear and I edited my question, but yes, I was interested to know if A would be more selected than two other items displayed next to it. The problem I had lied in the way I designed my experiment. Since three items were displayed to the subject, I was confused regarding the frequencies to use, and did not know if I had to include the frequencies in my statistical test. Reframing the question, I wanted to know if A would be chosen, whatever the two other items drawn and displayed next to it. $\endgroup$ – Jauhnax May 1 at 22:26
  • $\begingroup$ Then it seems that ad hoc tests of A vs. other single categories (as in my last prop.test) will do what you want. $\endgroup$ – BruceET May 2 at 0:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.