Hypothesis test based on entropy I am reading the wikipedia page on hypothesis testing, but a I can't find any reference to tests based on entropy. Which are good hypothesis tests based on entropy or quantities derived from it?
 A: A fantastic reference that I have been using for self study on this topic is Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach by Kenneth Burnham. In brief, a hypothesis test compares a test statistic, $T$, calculated from the data on hand (assumed to be from some distribution, usually Gaussian) to a critical value, e.g. $t_{crit}$ or $F_{crit}$ or $\chi^2_{crit}$ to establish the veracity of a hypothesis, e.g. $\mu=0$. 
Entropy, on the other hand, (better said, relative entropy) is an estimate of the expected Kullback-Leibler distance between the hypothesized model (could be Gaussian, Chi-square, whatever) and the true, generating model— i.e. nature— denoted by Burnham simply as $f$. What does estimated distance mean? Well, if the relative entropy, a.k.a. expected K-L distance between your candidate model, $g$, and full reality (nature) is large, then you have lost information (measured in bits) by using your candidate to try to represent reality. There is no hypothesis being tested in information-theoretic approaches, only (the distance from) truth to be discovered. 
The quantity used in statistics most often to measure this distance is the AIC. A quote from the textbook on the AIC: 

Thus, rather than having a simple measure of the directed distance between two models (i.e. the K-L distance), one has instead an estimate of the expected, relative distance between the fitted model and the unknown true mechanism (perhaps of infinite dimension) that actually generated the observed data. (Page 61)

A: To test for the significance of the difference between two entropy values, compute the maximum possible entropy, Log2N, then take the ratio with the actual entropy. The Z-test for proportions is an appropriate test.
