1
$\begingroup$

To compute the test statistic $\chi^2$ the sum is typically performed over all cells in a table, see e.g. https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test

The $\chi^2$-distribution is the distribution of a sum of the squares of k independent standard normal random variables.

So, I wonder why the sum to compute the test statistics is not only over the cells that are independent: sum over as many cells that there are degrees of freedom. E.g. in the case of a homogeneity test: just sum $(\text{row}-1)(\text{column}-1)$ cells.

I stumbled over this question by simulating the $\chi^2$-distribution for some examples from the corresponding null hypothesis. The simulated sample distribution fits the $\chi^2$-distribution better if I only perform the sum over independent cells. However, in textbooks the sum is over all cells.


Here is the simulation (Is constraint correctly implemented?).

Contingency table for the expectation, if X and Y are independent:

           Y
        0  |  1   |
   0  | 24 |  16  | 40
X  1  | 36 |  24  | 60
-------------------
        60   40     100
t=10 # 
data = np.array([[35,5],[25,35]]) *t # contingency table

# We want to test for independence, the expectation is
expected = np.array([[24,16],[36,24]]) *t

# simulation under the constraints
sim00 = np.random.binomial(n=100*t, p=24/100, size=10000000) 
simulation = np.zeros((10000000, 2, 2))
simulation[:,0,0] = sim00
simulation[:,1,0] = 60*t - sim00
simulation[:,:,1] = np.array([40*t, 60*t]) - simulation[:,:,0]

# sum over all cells 
sample_distribution = ((simulation[:,:,:]-expected[:,:])**2/expected[:,:]).sum(axis=1).sum(axis=1)
# the independet cell - you have to comment one out
sample_distribution = ((simulation[:,0,0]-expected[0,0])**2/expected[0,0])

# plot the simulation and the chi2 distribution
x = np.linspace(scipy.stats.chi2.ppf(0.3, df), scipy.stats.chi2.ppf(0.999, df), 100)
plt.plot(x, scipy.stats.chi2.pdf(x, df), 'r-', lw=2, alpha=0.6, label='$\chi^2$-pdf')
plt.hist(sample_distribution, bins=100, density=True, label='sampling')
plt.legend();

contribution of only one cell

contribution of all four cells


Update: The simulation (with the full sum) gives the expected result if the simulation is done with a hypergeometric_distribution.

# https://en.wikipedia.org/wiki/Hypergeometric_distribution 
# Working_example
nb_simulations=100000
s00 = np.random.hypergeometric(ngood=40*t, nbad=60*t, nsample=60*t, size=nb_simulations)
s = np.ndarray((nb_simulations, 2, 2))
s[:,0,0] = s00
s[:,0,1] = 40*t-s00
s[:,1,0] = 60*t-s00
s[:,1,1] = 60*t-s[:,1,0]
sample_distribution = ((s - expected)**2/expected).sum(axis=1).sum(axis=1)

enter image description here

$\endgroup$
7
  • 1
    $\begingroup$ I don't follow this. Can you paste in the simulation that motivates this question? $\endgroup$ Commented Dec 2, 2019 at 17:48
  • 4
    $\begingroup$ The simplest case of a $2\times 2$ table also presents the most extreme difference between the two sums--and there it is obvious that using only the value in one cell will not produce a chi-squared distribution. One is left suspecting there's a problem with your simulation. $\endgroup$
    – whuber
    Commented Dec 2, 2019 at 17:56
  • 1
    $\begingroup$ As it's currently written, the constraint is not being applied correctly. Only one cell is being generated randomly. For a 2x2 table under the null, two of the cells have to be generated randomly using the same probability of the outcome but conditioning on the observed sample size for each exposure, i.e. hold the sample size for that exposure constant. That is, you condition on the marginal totals of the rows or of the columns, but not both. $\endgroup$ Commented Dec 2, 2019 at 18:53
  • 1
    $\begingroup$ You may also need to calculate the expected values based on the observed data as you do with the Chi square test in practice. The code currently assumes the true expected values are known. $\endgroup$ Commented Dec 3, 2019 at 16:12
  • 1
    $\begingroup$ With the hypergeometric sampling, the calculated p(Y) of the simulated samples is the same as the true p(Y). That is the constraint. So the result is the same. $\endgroup$ Commented Dec 3, 2019 at 19:37

1 Answer 1

0
$\begingroup$

A simple sanity test: With $k$ cells and $k-1$ df (degrees of freedom), say, there are $k$ different ways of leaving out one cell from the sum. Which do you choose?

Besides that, after leaving out one cell, the remaining $k-1$ cells will not, typically, represent independent random variables.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.