chi-squared test for non-iid binomial data

Question

Simple case: Suppose we have $N$ observations of $X_i$ that we believe are theoretically iid binomial random variable $\text{Binomial}(k, p)$. Well, one way we could test to see if this is true is using the standard Pearson's chi-squared statistic.

The question: Suppose we have a more difficult problem, where we have $N$ different $X_i$'s, but now each is theoretically from $\text{Binomial}(k, p_i)$, with constant $k$ between all draws, and we know each $p_i$ corresponding to each observation.

Is there an obvious analytic extension of the previous test to this case? At the moment I am simulating a null distribution of chi-squared statistics from the known $p_i$'s and testing from that.

Answer 1

we have $N$ different $X_i$'s, but now each is theoretically from Binomial($k$,$p_i$), with constant $k$ between all draws, and we know each $p_i$ corresponding to each observation.

I don't think I am completely certain I understand the exact situation here. Do you have a small sample (even a made up one of the right structure) that will allow you to more clearly and specifically detail what you are actually observing? Is each 'draw' a single number or a vector, for example?

Is this that each draw at a given $p_i$ is seen, and the $p_i$ are known separately, or you only see some kind of sum of random variables?

If it's separate and the $p$'s are all known then you could calculate a chi-square on each component and add them. Your problem may be potentially small expected counts, but if you add many components this will be less of an issue. The alternative is to simulate the distribution at your particular combinations of counts and $p$s and so on.

An aside: Your question is somewhat related to this distribution:

http://en.wikipedia.org/wiki/Poisson_binomial_distribution

... but where you have not a Bernoulli for each component but a binomial.

Is there an obvious analytic extension of the previous test to this case? At the moment I am simulating a null distribution of chi-squared statistics from the known $p_i$'s and testing from that.

That seems quite sensible to me. You're simulating the exact distribution of a test statistic that you'd otherwise have to approximate.

While it could be approximated with a chi-square by just adding the individual components and their d.f., it seems that your approach - if the simulation size is sufficiently large - should be better.

You can check the accuracy of just using a chi-square pretty directly since you have a simulation already set up - just compare its distribution with the corresponding chi-square, paying particular attention to the regions near the kinds of significance level you're interested in (you probably don't care all that much if you calculate a p-value of 0.56 as 0.36 since i7t won't alter any decisions, but you probably care relatively more if you calculate one of 0.056 as 0.036, where it well could change your decision).