5
$\begingroup$

I want to compare two percentages on a single categorical variable from one sample. For example, in my data set, a variable can take on two values (i.e. A and B). Of 586 instances, 87.4% is in Category A (512 of 586) and 12.6% is in Category B (74 of 586).

  • Can I say 'my participants tend to be more concerned about Category A than B'?
  • Is a statistical test required to make such a claim? If so What statistical test is required?
$\endgroup$

2 Answers 2

8
$\begingroup$

Are you interested in being able to say that one of the percentages is greater than the other?

In the cases you want to do it, do they always add to 100%?

In that case, it's easy - you compare one of the percentages to 50%; if it's bigger than 50% the complementary one is smaller than 50%.

If you want to compare two proportions where there are other outcomes as well (say compare A and B but where there's a C, D and E), then you can take the A proportion of just the A's and B's and again, compare to 50%.

So these cases reduce to a one-sample-proportions test, for which there are many useful pages to be found by a search on that term (not only tutorial pages with worked examples but also videos), but let me give you a rough outline here of what calculations are involved.

It might help to think of it like a coin-toss. If I was tossing a fair coin 586 times, what proportions of 'heads' are reasonably likely to come up? If we painted an 'A' on the head-side and a 'B' on the other side we would record the proportion of 'A's we see when they're equally likely.

This is what the distribution of the proportion of heads looks like if we do that "toss a fair coin 586 times and count the proportion of heads (A's)" ... and then repeat the experiment ten thousand times:

binomial simulation

This shows that in your example of a total 586, if A's and B's are equally likely, the proportion of A's will nearly always be between 45% and 55%, and essentially never outside 40% to 60% (didn't happen once in ten thousand experiments).

If your sample size (586 in your example) is not too small - more than about 10-15 should be sufficient - you can use a normal approximation to the sample proportion.

If $\hat{p}$ is the sample proportion (512/586) and $p_0$ is the hypothesized "no difference" proportion (50%), you can calculate:

$z=\frac{\hat{p} - p_0}{\sqrt{p_0 (1-p_0)/n}}$

and use normal tables to calculate whether the difference is more than can be explained by chance.

In your example,

$z = \frac{(512/586 - 0.5)}{\sqrt{0.5 \times 0.5/586}} = 18.09$

You could very safely conclude that the proportion is different from 50% - and thereby higher than 50%, by inspection - and therefore it must logically also be higher than B. (It's possible to construct an explicit test of the difference in A and B proportions, but it gives the same result as the usual one sample test, so you might as well do the 'standard' thing.)

see http://en.wikipedia.org/wiki/Statistical_hypothesis_testing

At a typical significance level of 5%, you'd conclude that the A-proportion was different from 50% if $|z|$ is greater than 1.96 (at that sample size an A-count of 317 or more or 269 or less would lead you to conclude that the A proportion is different from 50%).

It's hard to tell how much of this you already know, so I'll let you ask questions to guide how much detail you need.

You don't need to obtain or learn sophisticated software for this test; you can easily do it by hand or in Excel (or a free equivalent like LibreOffice's Calc), for example, if you already have something like that. (If you want to try it, however, the package R is free and quite powerful, and very widely used by statisticians - so help on it is easy to find online.)

If you have Excel, you can calculate a p-value using NORMSDIST.

In small samples, you can't use the normal approximation and should use the exact binomial distribution, but again, that's reasonably straightforward to do in something like Excel.

[If your sample size is moderate - say between about 15 and 50 - I'd suggest using a continuity correction for the normal approximation, or using the exact binomial. Once you're past there (for a proportion of 50%), it usually doesn't make enough difference to bother with.]

$\endgroup$
4
  • $\begingroup$ Thank you so much for your answer. It's clear, but I'd like to check several things (I'm not good at stats). In my study, a total of 108 students participated. Each was asked to report 'lexical' or 'grammatical' problems during the task. I found that across the participants, 512 'lexical' problems were found, and 74 'grammatical' problems (a total of 586 problems). I reported the result with descriptive statistics (Mean(SD), and percentage), and I wrote 'lexical'(87.4%) was about seven times higher than 'grammatical'(12.6%)'. Should I use one-sample t-test in this case? Possible in SPSS? $\endgroup$
    – toeflzzang
    May 14, 2013 at 0:32
  • $\begingroup$ Uh, this is not the situation described in your original question. You really should have described your actual circumstances to begin with. With some additional (possibly unreasonable) assumptions, you might be able to use the analysis here, but I'd have recommended a different approach for what you now describe. Perhaps you should generate a new question with more details still, but a t-test is unlikely to be suitable. (If you have some experience of SPSS, you should probably also have mentioned that in your original question, as it alters the kinds of things we can say to do.) $\endgroup$
    – Glen_b
    May 14, 2013 at 0:47
  • $\begingroup$ I'm sorry for your confusion. My teacher suggested me to do one-sample t-test for this. I didn't know about the test at all, so, your answer (plus your explanation added later) was very helpful. I really appreciate it. I'll generate a new question with more details as you suggested. Thank you! $\endgroup$
    – toeflzzang
    May 14, 2013 at 0:55
  • $\begingroup$ In particular, what would happen if you had two kinds of students - some students who have a very high proportion of lexical problems and others who have a very high proportion of grammatical problems? You might conclude from this kind of 'put everyone in one bucket' test that the proportion of A's (prop$^n$. of lexical problems) was similar to the proportion of B's, when it isn't the case for even one of the students! This kind of issue - the possibility of strong variation in proportion across students - suggests you probably shouldn't just pool them all together as if they were homogeneous. $\endgroup$
    – Glen_b
    May 14, 2013 at 1:55
2
$\begingroup$

The "statistical test" your teacher is referring to would be a binomial test. It is an exact test, meaning that it yields the exact probability (p value) of your observed proportion (or a more extreme one) occurring under the null hypothesis. In this case, your null hypothesis appears to be that the proportion of people who pick category A (or alternatively, B) is .50.

For example, in R you would run the test as follows:

binom.test(x=512, n=586, p=.5, alternative="two.sided")

which yields p < .001, meaning that if it is the case that the true proportion in the population is .50, then the odds of finding a sample this size with this number of people picking category A (or an even more extreme proportion, i.e. a proportion closer to 1 or between 0 and 1-.874) is less than .001--a very small probability. You would conclude that people "tend indeed to be more concerned about Category A than B."

See here for more software options to run the test (e.g. SPSS, SAS).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.