# Is there any statistical significance test similar to chi-square that penalizes small groups, making them less significant?

Let's imagine an NxM contingency table, where rows represent groups and the columns represent attributes, so the table describes the number of entities with given attributes belonging into given groups. If my understanding is correct, the chi-square test only analyses ratios, i.e. it compares the ratio of entities with a given attribute in a given group to the of entities with the same attribute in other groups (observed ratio vs expected ratio). Thus, a very small group with a very high ratio will be a significant outlier. Is there a test or measure that also penalizes small groups and prioritizes large groups even with smaller ratios? Because, yes, I am interested in significantly higher ratios but from a practical usability, I am more interested in the larger groups.

• Your understanding seems to be wrong. I would not say the $\chi^2$-statistic of $\sum \frac{(O_i-E_i)^2}{E_i}$ "only analyses ratios". It can be rewritten as $\sum\left(\frac{O_i}{E_i}-1\right)^2 E_i$ which involves a ratio but "prioritizes large groups" thanks to the $E_i$ term Oct 31, 2023 at 17:06
• @Henry Yeah, I see now. I thought Oi and Ei are the ratios themselves, not "flat" values. In this case, small groups (small Ei) with large ratios (large Oi/Ei) will weigh less in the sum, if I'm not mistaken. Oct 31, 2023 at 17:35
• Smaller samples will more often have ratios away from $1$, but this is mitigated (not eliminated) by the $E_i$ term. Oct 31, 2023 at 17:52
• @oliver.c in the formula $\sum \frac{(O_i-E_i)^2}{E_i}$, the $O_i$ values have to be counts. Not ratios. Oct 31, 2023 at 22:23