When is a likelihood a likelihood?

Question

The probability density function is well characterized: it has to be measurable, and must integrate up to 1, and it must be non-negative over its support. In this case, the parameter vector is seen as fixed, and housed within a family of parametric distributions, like Exp($\theta$) or Weibull or...

For estimation and statistics, the concept of a likelihood is introduced. Roughly it is a function of the parameters rather than the data, since when we maximize the likelihood, we find the parameters that give the most "likely" set of data according to that family of parametric distributions rather than the other way around. However, as I understand it: the likelihood has no functional requirements as a function of $\theta$. Sure, it will be non-zero as it is the product of densities. But it may not add up to 1 if we integrate over $\theta$'s support (we would have to multiply by a prior to get that kind of Bayesian result).

We then get into these various forms of likelihood: a quasilikelihood, a pseudolikelihood, conditional likelihood, a partial likelihood, and so on and so forth. These respectively represent situations when we say, "Well... it's not a proper likelihood, but I'm going to maximize it and see what happens anyway." The delegation of such titles suggests that only when we know for certain that the probability model is appropriate for the data, the use of the term "likelihood" is warranted. But would that ever be the case in most practical scenarios? Surely that involves assumptions which are neither verifiable or interesting.

Take as an example the paired t-test. This is the maximum conditional likelihood. I don't care about the between pair mean differences, so by subtracting the paired observations, I can directly model the univariate within-pair mean differences. Why is it not possible to say I am using a normal probability model for the within pair differences? Can I not call such a thing a likelihood?

Is there a more practical or robust way of understanding maximum likelihood estimation? Should we not be calling everything a pseudo- or quasi- likelihood in all estimation and inference?

Answer 1

In a way, you are correct: all our models are wrong, hence even "exact" likelihoods are but convenient pseudolikelihoods to the likelihood function for the true underlying data process (assuming it could even be parameterized).

However, to understand likelihood, you need to move away from Dr. Box's adage of 'all models are wrong..." and live in a world where we pretend our model is correct. This means moving from applied to mathematical statistics.

In this more constrained context, the explanation of what is a likelihood is given by the definition of quasi and pseudo likelihood themselves:

A function $L$ is a likelihood iff it is developed using the true underlying distribution of the data.

Pseudolikelihood breaks with this definition by approximating $L$ using a different, but asymptotically correct, probability model. Quasilikelihood functions $Q$ represent an even further break from the definition of likelihood because they cannot be generated by any valid probability distribution. For example, if your data are iid, then:

$$\neg \exists P \in \mathcal{P}: L(\theta;x) = \prod_{i=1}^n P(x_i;\theta)$$