1
$\begingroup$

I've conducted a large survey with about 600 responders. Most of the questions were multiple choice questions. I'm interested in comparing several kinds of percentages. First, I'm simply interested in determining whether the percentage of people choosing answer A is the same or different from those choosing answer B, C, D. etc. Most of the online stuff I've read suggests this is a one sample test of proportions but those seem to require a theoretical or expected value to compare the percentage to and that's not the case here. My second broad category of analyses would be to compare BETWEEN populations. So, do males and females answer the questions differently. There I would be interested in seeing whether the array of answer choices differ (e.g. males chose A 40% of the time, B 50% of the time, and C 10% of the time while females chose A 30% of the time, B 10% of the time, and C 60% of the time) and whether specific percentages differ (e.g. is the percentage of males that chose A [40%] different from the percentage of females that chose A [30%]).

$\endgroup$
1
$\begingroup$

I am a little unsure of exactly what you want to do but when you are dealing with integer fractions you have a binomial distribution. I would make a simple binomial test by checking to what degree their clopper pearson confidence intervals overlap. Hope this helps.

You may also find this question helpful: Simultaneous Z-test for the equality of two proportions (binomial distribution)

$\endgroup$
  • $\begingroup$ These are multiple choice questions with 3 or 4 possible answers. There is not a binomial distribution here. I want to know for a given question if the number/percentage/proportion of responders who chose answer A is different from the number who chose B is different from the number who chose C, etc. $\endgroup$ – user52421 Jul 21 '14 at 22:25
  • $\begingroup$ The choice of A is yes or no, hence binomial. If you want something more complicated then multinomial. $\endgroup$ – Keith Jul 21 '14 at 22:28
  • $\begingroup$ so I would be comparing the number of people who chose A vs the number of people who didn't choose A (and thus chose B, C, or D) using a binomial test? $\endgroup$ – user52421 Jul 23 '14 at 1:19
  • $\begingroup$ I've found a program online that does a "one sample t-test between percentages" but there is very little documentation with program. have you heard of such a test? it seems to fit the bill here. $\endgroup$ – user52421 Jul 23 '14 at 3:16
  • $\begingroup$ While I know nothing about the program it is likely that it is using a binomial distribution under a Gaussian approximation. This may not be safe in your case. The conditions where it is are well documented so I will not say more. Assuming it is make sure it uses the Welch's t test. Such things are easy to find in standard software docs.scipy.org/doc/scipy/reference/generated/… $\endgroup$ – Keith Jul 23 '14 at 7:06
1
$\begingroup$

There are two types of comparisons you could make which I think correspond to your two queries.

Compare the responses to 2 questions (Q1, Q2):

|    | A   | B   | C   | D   |
| -- | --- | --- | --- | --- |
| Q1 | 150 | 150 | 150 | 150 |
| Q2 | 300 | 100 | 100 | 100 |

You would like compare whether question 1 and question 2 are being answered in the same way. Perform a chi-squared test.

Compare the responses to 1 question by group (G1, G2):

|    | A   | B   | C   | D   |
| -- | --- | --- | --- | --- |
| G1 | 150 | 150 | 150 | 150 |
| G2 | 300 | 100 | 100 | 100 |

You would like compare whether question 1 and question 2 are being answered in the same way. Again perform a chi-squared test!

There's an explanation of the method here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.