0
$\begingroup$

I have a dataset of 3,500,000 categorical observations that can be grouped by an ID. In these categories u occurs 25,000 times. How can I test to determine if a group has a different proportion of u compared with the whole dataset?

For example: a group has 2200 members where 300 have category u. What calculations would I make for this group?

Should I use a z-test or chi squared? Why/why not?

If I do this test for every group will I run into a multiple testing problem? Are there any other considerations I have missed?

$\endgroup$
1
  • 1
    $\begingroup$ It's best to compare the subset with its complement (sample of $2200,$ with $3,500,000 -2200$ others. One way is to make a $2 \times 2$ table with column headers Subset & Complement, column headers U & non-U, and appropriate counts in the four cells. Then do chi-squared test. // Another way is to compare proportion of U in subset with proportion of U in Complement using a z-test. $\endgroup$
    – BruceET
    Sep 3, 2018 at 23:57

1 Answer 1

0
$\begingroup$

In Minitab statistical software the printout for the z-test is shown below: Essentially, your are comparing $\hat p_S = 0.135$ in your sample with $\hat p_C = 0.007$ in the complement of the sample. Sample sizes are large enough to find that these proportions are significantly different.

The difference $\hat p_S - \hat p_C$ is approximately normally distributed. When you divide by an estimate of the standard deviation of this difference, you get a test statistic $Z$ that is normally distributed. Here $Z = -17.66$ and you can reject the null hypothesis that the two proportions are equal. I suppose you can find a formula for this test in a statistics textbook under 'Categorical Data: tests for two proportions'.

The last line of the output shows the results of Fisher's Exact test. You can find a description of that test on Wikipedia, if not in a textbook.

Test and CI for Two Proportions 

Sample      X        N  Sample p
1       25000  3497600  0.007148
2         300     2200  0.136364


Difference = p (1) - p (2)
Estimate for difference:  -0.129216
95% CI for difference:  (-0.143556, -0.114876)
Test for difference = 0 (vs ≠ 0):  Z = -17.66  P-Value = 0.000

Fisher’s exact test: P-Value = 0.000

If you do a test for independence in a two-by-two 'contingency table' as described in my Comment, you will reject the null hypothesis that categories Sample/Complement and U/non-U are independent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.