0
$\begingroup$

This is an excerpt from the wikipedia article on test statistics:

Suppose the task is to test whether a coin is fair (i.e. has equal probabilities of producing a head or a tail). If the coin is flipped 100 times and the results are recorded, the raw data can be represented as a sequence of 100 heads and tails. If there is interest in the marginal probability of obtaining a head, only the number T out of the 100 flips that produced a head needs to be recorded. But T can also be used as a test statistic in one of two ways:

  • the exact sampling distribution of T under the null hypothesis is the binomial distribution with parameters 0.5 and 100.

My interpretation of the first bullet: this saying that $T$ is a test statistic because we know that the distribution SHOULD be binomial with known parameters, so if the empirical distribution of T is too different than the binomial distribution, we would reject the null hypothesis that it's a fair coin.

If that interpretation is true, what would the mechanics of carrying out this test look like? In other words, how would we mark the threshold of what "too different" looks like when it comes to the empirical distribution and the binomial/null distribution?

$\endgroup$
0
$\begingroup$

Your test statistic could be related to the difference between the logarithms of the maximized null hypothesis likelihood (coin fair) and the maximized alternate hypothesis likelihood (coin not fair): $\lambda = 2(\ln \hat{L}_1 - \ln \hat{L}_0)$. The null likelihood would be a binomial with fixed probability $1/2$ (and 100 trials), the alternative one would be a binomial with free probability $p$ maximized to $\hat{p}$ for your data.

To reject the null hypothesis, you need to know the distribution of $\lambda$ under the null hypothesis. This may be possible using Wilk's theorem if the statistics are sufficient and under certain assumptions, or by running simulations. If the value of $\lambda$ you observe is large compared to the distribution, the null hypothesis is disfavored.

$\endgroup$
1
  • $\begingroup$ Okay that makes sense. Is there a distribution for cross entropy? Maybe I could compare the two distributions that way as well. It seems similar to what you are saying. $\endgroup$ – rocksNwaves Apr 2 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.