The p-value is the probability of obtaining test results at least as extreme as the results actually observed during the test, assuming that the null hypothesis is correct.

Why is the p-value a probability rather than a likelihood function?

  • $\begingroup$ I think that a p-value is more like a cumulative probability, whereas a likelihood is the probability of a single point (for discrete case) and P(X in (t,t+dt)) (for a continuous case). $\endgroup$ – M. Austin Jan 18 at 12:28
  • $\begingroup$ @M.Austin a likelihood (in the statistical definition) is not a probability. It is a function of parameter(s) given an observation. It may be non-negative and not guaranteed to sum to 1. As a simple example consider a Bernoulli(p). If $x=0$ observed integrating over $0<p<1$ gives you $1/2$ and not 1. $\endgroup$ – Lucas Roberts Jan 18 at 20:15

Likelihood applies to data once observed. If you were describing a property of this data set, you would be speaking of a likelihood. Probability describes a property of hypothetical data sets not yet observed. Against the arbitrary standard of this data set: the probability of observing another data set which departs from the null hypothesis as much as or more than this data set does.

  • $\begingroup$ when you say describing a property of the data set, a likelihood specifically gives you a function of the parameter(s), so the property would need to be a property defined in terms of (some function of) the parameter(s). $\endgroup$ – Lucas Roberts Jan 18 at 20:18

The term "probability function" refers to a function that takes a subset of the possibility space, and returns a real number. Thus, it is a measure on the possibility space, and fulfills all the axioms of a measure, with the additional restriction that the measure of the entire space be 1. The measure is fixed, and the output of the function changes as the subset changes.

The term "likelihood function" refers to a function that takes a hypothesis, and returns the probability under that hypothesis. Thus, the subset of the possibility space is fixed, and the measure changes (each hypothesis represents a different measure on the possibility space). Usually, the hypotheses are parameterized (often simplified down to one parameter), and the likelihood function is expressed as a function of those parameters.

For example, suppose you think that a statistic comes from a Poisson distribution. You could write the probability as $P_{\lambda}(x)$. That is, you're treating $\lambda$ as a fixed parameter that gives rise to a function of the variable $x$. Or you could write it as $P(x,\lambda)$. This notation treats $x$ and $\lambda$ as both being variables, and the probability being a function of the two. Or you could write it as $P_x(\lambda)$. This notation treats $x$ as a fixed parameter, and the probability being a function of $\lambda$.

The p-value isn't a function, so it's certainly not a likelihood function. However, it is the output of a likelihood function. When people distinguish between probabilities and likelihoods (without including the word "function"), they are distinguishing between things conceptualized as coming from a probability function versus a likelihood function. In standard hypothesis testing, you're finding the probability given the null hypothesis, so you're treating the hypothesis as fixed, and so you have a probability function (a function where the possibility subset is allowed to vary). The distinction isn't an intrinsic property of the number, it's a matter of what the number is being used for. If it's being used to decide whether to reject the null, it's a probability. If it's being used to find the greatest likelihood, it's a likelihood.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.