I am testing the agreement of two sets of data with each other, albit testing for the goodness-of-fit. However, both experimental data and model data has actually a lot of zeros in it so chi-squared test fails. I am a bit at lost now, what would be the correct fitness test for these kinds of scenario?

Note: Data are measurements of physical quantities rather than counts, so there will be a lot of fractional values.


Expected Actual
0        0
0        0
1        0.9

And there are about thousands of remaining data set not shown.

If we use chi-squared

$$ X^2=\sum\limits_j \dfrac{(O_j-E_j)^2}{E_j} $$

We will get undefined, since one of the elements is over zero.

  • $\begingroup$ I don't understand what you mean by "fails" or indeed even what problem you could be referring to. Can you explain further? $\endgroup$ – Glen_b -Reinstate Monica Feb 9 at 7:05
  • $\begingroup$ Updated to clarify further $\endgroup$ – Jones G Feb 9 at 7:11
  • $\begingroup$ How are you getting a fractional "Actual" count? Where are the expected numbers coming from? What sort of model is underlying these values? Additional context will help $\endgroup$ – Glen_b -Reinstate Monica Feb 9 at 7:43
  • $\begingroup$ It came from computer, let us say these are measure of distances. For example I want to test if an new specie of ant is actually moving in sine function when facing a threat. So I will be checking for fitness with sine curve. $\endgroup$ – Jones G Feb 9 at 7:57
  • $\begingroup$ How does that give an "actual" count that is not a whole number? How are the expected counts computed? $\endgroup$ – Glen_b -Reinstate Monica Feb 10 at 3:47

From comments:

These are not counts, but rather measurements of physical quantities,

The $\sum_i (O_i-E_i)^2/E_i$ formula is specifically for counts (in particular it relies on the way the variance works in the Poisson or the way variance and covariance work in the multinomial, which simplifies back to the Poisson formula). Don't use that statistic if you don't have counts, it's wrong (in that it doesn't have a chi-squared distribution under the null).

If your actuals ($O_i$) are Gaussian and you know the population variance of each actual ($\text{Var}(O_i)=V_i$), and all of the actuals are mutually independent, you might still be able to compute a chi-squared statistic of the form $\sum_i (O_i-E_i)^2/V_i$.

  • $\begingroup$ That seems to be promising, I know how to get the variance from a given complete sample, but I am not sure how to get $ V_i $. I will just be guessing but, could it be this one $$V_i = \sum_2^i (O_i - \bar{O})^2/(i - 1)$$ $\endgroup$ – Jones G Feb 11 at 11:40
  • $\begingroup$ I said 'population variance' and emphasized know ( rather than 'sample variance') quite deliberately. You won't have a chi squared test in the case of a denominator based on sample variances. It might be possible to construct an F test fir that case but it may be more sensitive to an assumption of normality. You may be better with some form of resampling a based test. $\endgroup$ – Glen_b -Reinstate Monica Feb 11 at 21:22
  • $\begingroup$ Further note the i subscript is an observation index rather than a count of the number of Os. $\endgroup$ – Glen_b -Reinstate Monica Feb 11 at 21:26

Have you considered KL divergence : https://en.m.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence once data is transformed into a frequency table (sample pdf) . Actually once this transformation is done and if actual data follows t distribution , u can use chi squared and t test too ?

  • $\begingroup$ I have not, but when I looked at its definition, it seems to also give undefined result, since ln(0/0) is undefined. Or am I wrong? $\endgroup$ – Jones G Feb 9 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.