# Test whether probabilities of heads are under-estimated across many coins

I am trying to figure out a problem that is equivalent to the following. Suppose you have a bag of $$n$$ coins in which each coin is labelled with a probability $$p_i$$ that it will come up heads when you toss it. You are given the result of $$m$$ coin-flips as well as which coin was flipped (hence the relevant $$p_i$$s each time. What I am do is test the hypothesis that the labels on the coins underestimate the probability of heads coming up.

At first I wanted to take a very simple approach to this and just simulate things but the results seem strange to me and I think something must be wrong. What I did was simulate $$N$$ sets of size $$m$$ draws using a fixed set of probabilities for each coin. I then calculated the likelihood of each of these draws using the probabilities given on the coins. Then, I drew a set 'test set' of coins from a with a different set of probabilities (in practice I set the probabilities of the 'test-set' of coins to be the values of the original set +0.2), calculated the likelihood and compared this to the 10000 simulations. When I do this I often find that the likelihood of the test sample is not different to that of the true data. While this is obviously totally possible, when I just run a fischer exact test on the failures and successes for the same simulation (or even just a binomial hypothesis test), the data comes out as different. Is this possible/likely? Am I doing something wrong? Is there an alternative way to do this hypothesis test?

If you toss $$m$$ coins then the number of heads follows a $$X_i\sim B(1, \pi_i)$$ distribution. Therefore the test statistics $$X=\sum_{i=1}^m X_i$$ follows a Poisson binomial distribution.
If $$p_0=\sum_{i=1}^m p_i$$ and $$\pi$$ is the true sum of probabilities then you could test $$H_0: \pi\leq p_0$$ vs. $$H_1: \pi>p_0$$ (underestimation case). There are efficient methods to compute $$Pr(X=k)$$ such that you can do even an exact test.