# Statistical Test for one-sample, multiple choice option from a survey

I have conducted a survey where 410 people were given 14 brands. Of those 14, they were asked, "Which brand do you consider to be the "#1 Trust Brand for Protection". Respondents could only choose one brand for their answer.

I have the frequency and proportion results for the brands selected (using Stata)

      Choice           Freq.    Perc.       Cum.
Brand 1|         70       17.07       17.07
Brand 2|         64       15.61       32.68
Brand 3|         62       15.12       47.80
Brand 4|         46       11.22       59.02
Brand 5|         27        6.59       65.61
Brand 6|         25        6.10       71.71
Brand 7|         24        5.85       77.56
Brand 8|         22        5.37       82.93
Brand 9|         18        4.39       87.32
Brand 10 |         15        3.66       90.98
Brand 11 |         14        3.41       94.39
Brand 12 |          9        2.20       96.59
Brand 13 |          9        2.20       98.78
Brand 14 |          5        1.22      100.00


I am tasked with determining if Brand 1 is statistically significantly higher than Brand 2 (and then eventually all other individual brands).

I have made each Brand its own binomial variable (i.e. 0/1 for each brand, if selected)

Here are my constraints for identifying the appropriate statistical test:

• There is only one "group" (the n=410 sample). I'm not testing between any different groups
• I know that this is a one-sample test, because it's the same respondent across the variables.
• The test has to be appropriate for binomial variables (i.e. Brand 1 select 0/1).

I feel like I can't do a one-sample proportion test (such as prtest Brand1 = 0.0714 ) because I know the brands are NOT equally likely to be selected, since more popular brands are more likely to be chosen (I have a mix of popular and unknown brands).

I have followed this website (https://www.ssc.wisc.edu/sscc/pubs/sfs/sfs-prtest.htm) and essentially did prtest Brand1 == Brand2), knowing that this really isn't 2 sample. Is that proper?

I don't think a t-test is right because the brand variables (0/1) aren't independent (since they come from the same respondent). I don't think a paired t-test or McNemar is right (no before/after manipulation) or a chi-square is correct (such as tabulate Brand1 Brand2, chi2) because I don't have an expected distribution or outcome.

Can anyone help me understand what indeed is the right statistical test? I'm stuck because I don't have an expected frequency or proportion.

• This problem is very amenable to a brute-force Bayesian approach, i.e. treat it as 14 separate binomial estimation problems where you calculate a credible interval for the binomial parameter for the incidence for each brand, and results from brands with intervals that don't overlap are statistically significant from one another. But they're not making you use frequentist methods, are they? Commented Jun 14, 2021 at 1:28
• Hi @NaiveBayesian - no one has dictated any specific test to me. To your point, I've looked at CIs. 100% right if the CIs don't overlap, then there's a statistically significant difference. However, in some cases, where the brands' CIs barely overlap, it's my understanding that this overlap does not necessarily indicate that the means are not statistically different. Is that a proper understanding? (See "Mistake #1: link ). I will admit that I'm not versed in Bayesian methods. Commented Jun 14, 2021 at 11:48
• You are correct, just looking at the overlap is definitely a conservative test. On the other hand, although it's still taught in the textbooks, problems with the use of p-values as the final word on statistical significance are being widely acknowledged. Note especially point 6 here, though for a simple binomial model like this, a p-value based test might be reasonable. I will post a more robust method, as a full answer below. Commented Jun 15, 2021 at 2:46