# How do I test if a proportion is significantly different from expected?

I have data that falls under three categories. Using a Chi-squared goodness of fit test, I have a P-Value of <.001, so my proportions do not fit within the expected numbers.

Observed    Observed proportion    Expected Proportion
70751       .678107                .454545
30036       .287878                .454545
3549        .034015                .090909


What I want to do is test to see if the first category's observed proportion is significantly higher than its expected value. Essentially, I want to test if the first category is over-represented in the data to a statistically significant degree. How would I do that?

You could do an exact multinomial test, using as $H_0$ that the true proportions vector equals your expected one $\mathbf{p}_0 = (0.45,0.45,0.09)$. The P-value is then the probability of your observed proportion or more extreme under this null hypothesis. This is an omnibus test that tests all proportions, taking into account their dependencies.

An alternative is to do an exact binomial test (or a proportions t-test) only for the first category. However this may measure an "indirect" or "compositional" effect, whereby the proportion of the first category changes because of changes in the other categories. Judge for yourself if this is a problem for your case or not. If yes, look into "compositional data analysis" whereby ratios of proportions are of interested rather than absolute differences.

To me it feels like for this sample size the difference will be very significant anyhow.

Since the sample size is so huge, you can actually do a very intuitive test based on Chebyshev's inequality.

Standard deviation for the first category is $\sqrt{np(1-p)}$=160.8. So the observation is 145 standard deviations away from the expected value of $np = 47425.4$.

Chebyshev's inequality says that the probability of a random variable being $k$ or more standard deviations away from its mean is at most $1/k^2$. So that shows the p value must be $\le 1/145^2 = .0000475$.

I don't know how you decided to look at the first category, so you may have a multiple testing issue. However, this is easily solved with the Bonferroni correction: if we do all three tests, multiply all of the p values by 3. The adjusted p value for the first test is then $\le .000143$.

Normally I would never use the Bonferroni correction, and I've never seen anyone use Chebyshev's inequality for p values, just because these methods are so inefficient and conservative. But they are also extremely simple mathematically. When sample sizes are enormous like this, there's something I like aesthetically about using such simple reasoning to show statistical significance.

• Is it really true that the marginal standard deviation of the first category is sqrt(np(1-p)) ? That's ignoring the covariance structure of a multinomial RV. – khol Apr 26 '18 at 15:27
• Yes. See the wikipedia page. – user54038 Apr 26 '18 at 15:33
• I think that you should make it more clear that Chebyshev gives an upper bound for the probability, and thus the p value is at most .0000475. – Acccumulation May 1 '18 at 18:03
• Edited to make it clear that Chebyshev is providing upper bounds on p values. – user54038 May 3 '18 at 3:26