4
$\begingroup$

I have a table with the following format

        CountH SubcountH1 SubcountH2 CountK...
VarA
SubA1
SubA2
SubA3
VarB
SubB1
SubB2
.
.
.

The table is filled with numbers. My boss suggested in my last meeting to try to find statistical relationships between VarA and VarB. I am not sure how to do this (or even if it is possible) with an aggregated table like this. For example, the cell (SubA1, Subcount1) has the total number of observations of the variable SubA1 with its counts broken down by another variable (think of A to be income with three levels and the counts be Male and Female, say).

As you can tell, there is a lot of redundancies as the rowise-sum of the two columns SubcountH1 and SubcountH2 equals the CountH column (this would be similar for other column-variables H,K...). Similar situation with the rows (SubA1+SubA2+SubA3 add up to VarA, etc). Further, (VarX, Count) is pretty much the same for every variable X (as this is the total count for variable X).

Is there any bibliographical resource I could check to handle aggregated data tables like this?

I believe this table can only be used to make plots and some data visualisation but not statistical inference, is my intuition correct?

If my question is unclear, please ask me so I can rephrase it accordingly. Thank you for your time.

$\endgroup$

1 Answer 1

4
$\begingroup$

Contingency table tests (e.g., the $G$ test, the $\chi^{2}$ test) can often test for evidence of difference/evidence of association (and also equivalence, though those are more sophisticated) in the probability distributions of categories of rows by categories of columns in tables of counts (i.e. contingency tables).

A simple example is the 2×2 table (the 2×2 being the observed counts, $O_{ij}$):

Group A Group B Marginal Total
Count of Positive Count of Positive No. Positive in A + No. Positive in B
Count of Negative Count of Negative No. Negative in A + No. Negative in B
Total Count in A Total Count in B Total

$\boldsymbol{G}$ Test:

  1. $\text{H}_{0}\text{: }P(\text{Positive}|\text{Group = A}) = P(\text{Positive}|\text{Group = B})$, $\text{H}_{\text{A}}\text{: }P(\text{Positive}|\text{Group = A}) \ne P(\text{Positive}|\text{Group = B})$

    This null hypothesis implies that the best estimate of $P(\text{Positive})$ combines counts from both groups, or (No. Positive in A + No. Positive in B) ÷ Total. When multiplied by the total counts in each group, these create the expected counts, $E_{ij}$ for the top row in a table of expected counts. For example, the expected count of positives for Group A = Total Count in A × (No. Positive in A + No. Positive in B) ÷ Total and (No. Negative in A + No. Negative in B) ÷ Total gives $P(\text{Negative})$ under the null, which does similar for the bottom row in the table of expected counts:

Group A Group B Marginal Total
Expected Count of Positive Expected Count of Positive No. Positive in A + No. Positive in B
Expected Count of Negative Expected Count of Negative No. Negative in A + No. Negative in B
Total count in A Total count in B Total
  1. The test statistic $G = 2\sum_{ij}O_{ij}\ln\left(\frac{O_{ij}}{E_{ij}}\right)$, where $i,j$ index the cells across rows and columns.

  2. The probability of rejecting $\text{H}_{0}$, assuming it is true, is $p = P(G_{\text{df}=1}\ge G)$, where $\text{df} = \text{No. rows}-1 \times \text{No. columns} - 1$, and the $p$ value is based on the $\chi^{2}$ distribution with $\text{df}$ degrees of freedom.

  3. Reject $H_{0}$ if $p\le \alpha$.

  4. If you rejected $\text{H}_0$, conclude that you found evidence of a difference in probability of positives by group/found that probability of positive is associated with group. If you did not reject $\text{H}_0$, conclude that you failed to find such evidence.

This test extends to contingency tables of arbitrary number of rows and columns, although the null becomes "No association between row variable and column variable," with the alternative "Association between row variable and column variable."

$\boldsymbol{\chi^2}$ test

As above, but the test statistic is $\chi^{2} = \sum_{ij} \frac{\left(O_{ij}-E_{ij}\right)^{2}}{E_{ij}}$, and the probability of rejecting is $p = P(X^{2}_{\text{df}} \ge \chi^{2})$.

There are a few nuances (e.g., continuity corrections for 2×2 tables, corresponding equivalence tests and their hypotheses), but that is the meat of it. You will want to make your rows correspond to a single main category or sub-category (rather than use categories and subcategories), and will need to organize your columns by group (A, B, C, etc.).

$\endgroup$
1
  • 2
    $\begingroup$ Thank you so much for this answer. In fact, your answer made me remember that I read something related to contingecy tables (Wasserman's All of Statistics ch 15) some time ago. $\endgroup$
    – William M.
    Commented Sep 13, 2021 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.