Return to Question

Is there an example where not adhering to the likelihood principle (LP) would lead one to markedly different (and equally defensible) inferences with two defensible procedures that have proportional likelihoods?

All the examples I see are very silly, comparing a binomial with a negative binomial, where the p-value of the first is 7% and of the second 3%, which are "different" only insofar one is making binary decisions on arbitrary thresholds of significance such as 5%. If I change the threshold for 1%, for instance, both lead to the same conclusion.

I've never seen an example where it would lead to markedly different and defensible inferences. Is there such an example?

I'm asking because I've seen so much ink spent on this topic, as if the LP is something fundamental in the foundations of statistical inference. But if the best example one has are silly examples like the one above, the principle seems completely inconsequential.

Thus, I'm looking for a very compelling example, where if one does not follow the LP the weight of evidence would be overwhelmingly pointing in one direction given one test, but, in a different test with proportional likelihood, the weight of evidence would be overwhelmingly pointing in an opposite direction, and both conclusions look sensible.

Ideally, one could demonstrate we can have arbitrarily far apart, yet sensible, answers, such as tests with $p =0.1$ versus $p= 10^{-10}$ with proportional likelihoods and equivalent power to detect the same alternative.

PS: Bruce's answer does not address the question at all.

Is there an example where two different defensible tests with proportional likelihoods would lead one to markedly different (and equally defensible) inferences, for instance, where the p-values are order of magnitudes far apart, but the power to alternatives is similar?

All the examples I see are very silly, comparing a binomial with a negative binomial, where the p-value of the first is 7% and of the second 3%, which are "different" only insofar one is making binary decisions on arbitrary thresholds of significance such as 5% (which, by the way, is a pretty low standard for inference) and do not even bother to look at power. If I change the threshold for 1%, for instance, both lead to the same conclusion.

I've never seen an example where it would lead to markedly different and defensible inferences. Is there such an example?

I'm asking because I've seen so much ink spent on this topic, as if the Likelihood Principle is something fundamental in the foundations of statistical inference. But if the best example one has are silly examples like the one above, the principle seems completely inconsequential.

Thus, I'm looking for a very compelling example, where if one does not follow the LP the weight of evidence would be overwhelmingly pointing in one direction given one test, but, in a different test with proportional likelihood, the weight of evidence would be overwhelmingly pointing in an opposite direction, and both conclusions look sensible.

Ideally, one could demonstrate we can have arbitrarily far apart, yet sensible, answers, such as tests with $p =0.1$ versus $p= 10^{-10}$ with proportional likelihoods and equivalent power to detect the same alternative.

PS: Bruce's answer does not address the question at all.

Is there an example where not adhering to the likelihood principle (LP) would lead one to markedly different (and equally defensible) inferences with two defensible procedures that have proportional likelihoods?

All the examples I see are very silly, comparing a binomial with a negative binomial, where the p-value of the first is 7% and of the second 3%, which are "different" only insofar one is making binary decisions on arbitrary thresholds of significance such as 5%. If I change the threshold for 1%, for instance, both lead to the same conclusion.

I've never seen an example where it would lead to markedly different and defensible inferences. Is there such an example?

I'm asking because I've seen so much ink spent on this topic, as if the LP is something fundamental in the foundations of statistical inference. But if the best example one has are silly examples like the one above, the principle seems completely inconsequential.

Thus, I'm looking for a very compelling example, where if one does not follow the LP the weight of evidence would be overwhelmingly pointing in one direction given one test, but, in a different test with proportional likelihood, the weight of evidence would be overwhelmingly pointing in an opposite direction, and both conclusions look sensible.

Ideally, one could demonstrate we can have arbitrarily far apart, yet sensible, answers, such as $p =0.1$ versus $p= 10^{-10}$ with proportional likelihoods.

PS: Bruce's answer does not address the question at all.

Is there an example where not adhering to the likelihood principle (LP) would lead one to markedly different (and equally defensible) inferences with two defensible procedures that have proportional likelihoods?

All the examples I see are very silly, comparing a binomial with a negative binomial, where the p-value of the first is 7% and of the second 3%, which are "different" only insofar one is making binary decisions on arbitrary thresholds of significance such as 5%. If I change the threshold for 1%, for instance, both lead to the same conclusion.

I've never seen an example where it would lead to markedly different and defensible inferences. Is there such an example?

I'm asking because I've seen so much ink spent on this topic, as if the LP is something fundamental in the foundations of statistical inference. But if the best example one has are silly examples like the one above, the principle seems completely inconsequential.

Thus, I'm looking for a very compelling example, where if one does not follow the LP the weight of evidence would be overwhelmingly pointing in one direction given one test, but, in a different test with proportional likelihood, the weight of evidence would be overwhelmingly pointing in an opposite direction, and both conclusions look sensible.

Ideally, one could demonstrate we can have arbitrarily far apart, yet sensible, answers, such as tests with $p =0.1$ versus $p= 10^{-10}$ with proportional likelihoods and equivalent power to detect the same alternative.

PS: Bruce's answer does not address the question at all.