I am a bit unsure in my approach of applying the Chi2 GOF test to my data and calculating the correct effect size.
The situation: I am studying the occurrence of specific biological phenomena in certain regions of the genome. My hypothesis is, that the phenomena do not occur randomly but are enriched in certain regions. The data I gathered looks like this:
|------------|-------------------|
| Region | Observed counts |
|------------|-------------------|
| A | 673 |
|------------|-------------------|
| B | 118 |
|------------|-------------------|
| C | 1001 |
|------------|-------------------|
| D | 1066 |
|------------|-------------------|
| E | 91 |
|------------|-------------------|
As regions A-E are not identical in size, I would not expect equal counts in each region randomly. Instead I calculated relative sizing factors and derived the expected, random distribution:
|------------|-------------------|-------------------|
| Region | Expected ratios | Expected counts |
|------------|-------------------|-------------------|
| A | 0.050 | 146 |
|------------|-------------------|-------------------|
| B | 0.026 | 78 |
|------------|-------------------|-------------------|
| C | 0.347 | 1024 |
|------------|-------------------|-------------------|
| D | 0.554 | 1635 |
|------------|-------------------|-------------------|
| E | 0.022 | 66 |
|------------|-------------------|-------------------|
The resulting contigency table looks like this (however I am unsure, if this is really a true contigency table):
|------------|-------------------|-------------------|
| Region | Observed counts | Expected counts |
|------------|-------------------|-------------------|--
| A | 673 | 146 | 819
|------------|-------------------|-------------------|--
| B | 118 | 78 | 196
|------------|-------------------|-------------------|--
| C | 1001 | 1024 | 2025
|------------|-------------------|-------------------|--
| D | 1066 | 1635 | 2701
|------------|-------------------|-------------------|--
| E | 91 | 66 | 157
|------------|-------------------|-------------------|--
| 2949 | 2949 |
To check, if the observed distribution deviates from what I would expect by chance, I compared both distributions using the Chi2 GOF test (chisq.test in R):
chi2 <- chisq.test(x = observed, p = expected/sum(expected))); chi2
> Chi-squared test for given probabilities
> data: observed
> X-squared = 2130.8, df = 4, p-value < 2.2e-16
As the n of my observations (n=2949) is, I believe, straining the informative value of the Chi2 test’s p-value, I want to include a measurement of effect size. As my contigency table is not 2x2 I am using Cramer’s V instead of Phi by applying the following formula:
V = sqrt(X2/n*df)
df = min(n_rows-1, n_cols-1) = 4
This results in a value of 0.4250119, indicating a strong difference between the observed and expected distribution and further authenticating the rejection of the H0 of no difference.
Questions: Is this approach valid? I am especially unsure about the creation of the contingency table and correspondingly the calculation of the dfs. Also, is there a way to further elaborate on the contribution of the different regions towards the difference between both contributions? Intuitively I see that the non-random effect is mainly due to an increase in region A and a decrease in Region D, but I’m not sure how to quantify this within the test.
I hope I could make my point clear. Cheers and thanks in advance!
Edit: reply to Bruce's answer
1. Calculation of expected counts/meaning of sizing factors The phenomena I am studying can only occur at certain positions in the genome. The sizing factors reflect the number of all possible positions within each region. They are simply the ratio positions_in_region/total_number_of_positions_in_genome and are therefore affected by the absolute size of the regions and the density of the positions within. The expected counts thus represent the number of occurences I would expect, if each individual position had the same probability to be affected by the studied phenomenon independent from the region it is found in.
2. The 'contigency table' I was aware that my table did not really fit the description of a contingency table and this was a major reason for my doubts of the correctness of my approach. I am glad that you clarified the difference. Also the totals of each category are clearly useless as pointed out correctly.
3. Distinct difference in specific regions/Pearson residuals As you showed using the Pearson residuals, the high chi2-statistic is driven mainly by a reduction of counts in region D and an increase in region A. This absolutely matches my general hypotheses, as region A is functionally the most relevant region, while region D is largely irrelevant. Is there a way to present those residuals in a standardized manner (e.g. relative contribution to the difference)? Especially interesting to me is also that region C apparently shows a count very similar to the one you would expect randomly. Is it possible to associate significance measurements to the individual residuals to make a point about the (non-)randomnes in the individual regions?