# Likelihood ratio test vs. $\chi^2$/Z-test for comparing binomial datasets

When does/can one use the likelihood ratio significance test instead of Fisher's exact test or its Pearson $\chi^2$ approximation for comparing two binomial datasets?

Given two binomial datasets (distributions), I'm seeing the LR test being used to compare one distribution against the global (combined) distribution. Usually I apply Fisher's test for comparing one dataset against the other. I realize that LR testing is of the Neyman-Pearson school, which assumes a fully specified alternative model as well as null model. E.g., in the LR test Wikipedia page example, it's being used to compare two binomial datasets (# heads/tails for two coins).

Why not use the $\chi^2$ test to compare the two samples against each other? What are the conceptual differences in these two approaches? When do I use which? And when is it appropriate to compare one sample against not its complement but the global dataset?

Generally speaking, the likelihood ratio and the ordinary Pearson $\chi^2$ tests are more accurate than Fisher's "exact" test. But for your situation you need an extremely heavy multiplicity adjustment thrown in, not matter which statistical test is used. Decision trees such as the one you are building require amazingly large datasets for their structure to validate. In a quick look at the CN2 link you provided I could not tell if the algorithms incorporated shrinkage (panelization; regularization). If not, watch for over-interpretation.
• Could you elaborate on the relative merits/appropriateness of LR vs. Pearson $\chi^2$ (which is just an approximation of Fisher's exact test)? And why are these more accurate than Fisher's exact test? – Yang May 4 '11 at 19:39
• Nothing in what you wrote provides any indication that the method is safe from overinterpretation (a form of overfitting). One reason I worry about the method is that it doesn't treat continuous variables continuously. The problem with Fisher's test is that the P-values are too large. See for example Crans GG Stat in Med 27:3598; 2008. The word "exact" has misled practitioners for decades. The ordinary Pearson $\chi^2$ is usually more accurate. – Frank Harrell May 4 '11 at 23:57
• Interesting - sounds like a relatively recent result that directly contradicts I think several sources I've come across over time that recommend using Fisher over $\chi^2$. Unfortunately I don't have access to that paper. What about the likelihood ratio test, and what about comparing one dataset against the combined dataset - when should I use these? – Yang May 5 '11 at 1:01