Creation of contigency table and calculation of effect size (Chi2, Cramer's V)

Question

I am a bit unsure in my approach of applying the Chi2 GOF test to my data and calculating the correct effect size.

The situation: I am studying the occurrence of specific biological phenomena in certain regions of the genome. My hypothesis is, that the phenomena do not occur randomly but are enriched in certain regions. The data I gathered looks like this:

       |------------|-------------------|
       |   Region   |  Observed counts  |
       |------------|-------------------|
       |     A      |        673        |
       |------------|-------------------|
       |     B      |        118        |
       |------------|-------------------|
       |     C      |        1001       |
       |------------|-------------------|
       |     D      |        1066       |
       |------------|-------------------|
       |     E      |        91         |
       |------------|-------------------|

As regions A-E are not identical in size, I would not expect equal counts in each region randomly. Instead I calculated relative sizing factors and derived the expected, random distribution:

       |------------|-------------------|-------------------|
       |   Region   |  Expected ratios  |  Expected counts  |
       |------------|-------------------|-------------------|
       |     A      |        0.050      |        146        |
       |------------|-------------------|-------------------|
       |     B      |        0.026      |        78         |
       |------------|-------------------|-------------------|
       |     C      |        0.347      |        1024       |
       |------------|-------------------|-------------------|
       |     D      |        0.554      |        1635       |
       |------------|-------------------|-------------------|
       |     E      |        0.022      |        66         |
       |------------|-------------------|-------------------|

The resulting contigency table looks like this (however I am unsure, if this is really a true contigency table):

       |------------|-------------------|-------------------|
       |   Region   |  Observed counts  |  Expected counts  |
       |------------|-------------------|-------------------|--
       |     A      |        673        |        146        |  819
       |------------|-------------------|-------------------|--
       |     B      |        118        |        78         |  196
       |------------|-------------------|-------------------|--
       |     C      |        1001       |        1024       |  2025
       |------------|-------------------|-------------------|--
       |     D      |        1066       |        1635       |  2701
       |------------|-------------------|-------------------|--
       |     E      |        91         |        66         |  157
       |------------|-------------------|-------------------|--
                    |        2949       |        2949       |

To check, if the observed distribution deviates from what I would expect by chance, I compared both distributions using the Chi2 GOF test (chisq.test in R):

chi2 <- chisq.test(x = observed, p = expected/sum(expected))); chi2
> Chi-squared test for given probabilities
> data:  observed
> X-squared = 2130.8, df = 4, p-value < 2.2e-16

As the n of my observations (n=2949) is, I believe, straining the informative value of the Chi2 test’s p-value, I want to include a measurement of effect size. As my contigency table is not 2x2 I am using Cramer’s V instead of Phi by applying the following formula:

V = sqrt(X2/n*df)
df = min(n_rows-1, n_cols-1) = 4

This results in a value of 0.4250119, indicating a strong difference between the observed and expected distribution and further authenticating the rejection of the H0 of no difference.

Questions: Is this approach valid? I am especially unsure about the creation of the contingency table and correspondingly the calculation of the dfs. Also, is there a way to further elaborate on the contribution of the different regions towards the difference between both contributions? Intuitively I see that the non-random effect is mainly due to an increase in region A and a decrease in Region D, but I’m not sure how to quantify this within the test.

I hope I could make my point clear. Cheers and thanks in advance!

Edit: reply to Bruce's answer

1. Calculation of expected counts/meaning of sizing factors The phenomena I am studying can only occur at certain positions in the genome. The sizing factors reflect the number of all possible positions within each region. They are simply the ratio positions_in_region/total_number_of_positions_in_genome and are therefore affected by the absolute size of the regions and the density of the positions within. The expected counts thus represent the number of occurences I would expect, if each individual position had the same probability to be affected by the studied phenomenon independent from the region it is found in.

2. The 'contigency table' I was aware that my table did not really fit the description of a contingency table and this was a major reason for my doubts of the correctness of my approach. I am glad that you clarified the difference. Also the totals of each category are clearly useless as pointed out correctly.

3. Distinct difference in specific regions/Pearson residuals As you showed using the Pearson residuals, the high chi2-statistic is driven mainly by a reduction of counts in region D and an increase in region A. This absolutely matches my general hypotheses, as region A is functionally the most relevant region, while region D is largely irrelevant. Is there a way to present those residuals in a standardized manner (e.g. relative contribution to the difference)? Especially interesting to me is also that region C apparently shows a count very similar to the one you would expect randomly. Is it possible to associate significance measurements to the individual residuals to make a point about the (non-)randomnes in the individual regions?

Answer 1

I would like to know more about the procedure for computing 'expected counts' (leading to probabilities). Everything else rests on those computations, so it is crucial that you have an iron-clad rationale for them. [What do you mean by 'sizing factors'? The totals in the last column of your third table appear to be meaningless.]

Taking observed and expected counts as given, I verified your output from chisq.test and computed the chi-squared statistic separately. You have the correct degrees of freedom (5 categories minus 1).

You do not have a 'contingency table' here. That would be a data structure with rows and columns giving observed counts for two categorical variables. You have one categorical variable with five levels (regions) with observed counts and hypothetical probabilities for each level. My question above amounts to wondering how you got the hypothetical probabilities.

My next step would be to notice that regions A, B, and to an extent E, have higher observed counts than expected counts. By contrast, region C has about the expected number of observed counts, and region D has many fewer than the expected numbers of observed counts. Can you explain the these differences among the various regions? Are these the differences you anticipated before you did the experiment?

The 'so-called' Pearson residuals are as follows:

abs(obs - exp)/sqrt(exp)
[1] 43.614830  4.529108  0.718750 14.071921  3.077287

It is usually worthwhile reflecting on the very largest Pearson residuals (here for regions A and D). The sum of the squares of the Pearson residuals is the chi-squared test statistic.

With the very high chi-squared statistic (consequently tiny P-value) and noticeable discrepancies between observed and expected counts, it seems obvious without further analysis that you have differences that are highly statistically significant. The main issue is to explain on a practical level what the significant effect means and what may account for it.

Answer 2

Assuming the calculation of the expected proportions makes sense, it looks like you have a classic set-up for a chi-square goodness-of-fit test. You can calculate a variant of Cramer's V as you have suggested. But there are a few caveats. First, I can't find any good reference for doing this. Second, while the statistic will be bound by zero for a perfect match of observed and expected values, it will be bound by 1 only if the expected proportions are all equal. This may make interpretation of this statistic difficult in cases where the the expected proportions are not equal. For your expected proportions, I got a maximum V value of 3.305.

Also note that your formulae have a couple of errors. V = sqrt(X2/n*df) should be V = sqrt(X2/n/df) or V = sqrt(X2/(n*df)). Also, df = min(n_rows-1, n_cols-1) would result in an answer of 0, since there's only one column. Better just to think of it as K - 1.

There is a function, cramerVFit that calculates this statistic in R in the rcompanion package. (Caveat, I am the author of this package). It agrees with your result.