# What is the difference between a partial likelihood, profile likelihood and marginal likelihood?

I see these terms being used and I keep getting them mixed up. Is there a simple explanation of the differences between them?

The likelihood function usually depends on many parameters. Depending on the application, we are usually interested in only a subset of these parameters. For example, in linear regression, interest typically lies in the slope coefficients and not on the error variance.

Denote the parameters we are interested in as $$\beta$$ and the parameters that are not of primary interest as $$\theta$$. The standard way to approach the estimation problem is to maximize the likelihood function so that we obtain estimates of $$\beta$$ and $$\theta$$. However, since the primary interest lies in $$\beta$$ partial, profile and marginal likelihood offer alternative ways to estimate $$\beta$$ without estimating $$\theta$$.

In order to see the difference denote the standard likelihood by $$L(\beta, \theta|\mathrm{data})$$.

Maximum Likelihood

Find $$\beta$$ and $$\theta$$ that maximizes $$L(\beta, \theta|\mathrm{data})$$.

Partial Likelihood

If we can write the likelihood function as:

$$L(\beta, \theta|\mathrm{data}) = L_1(\beta|\mathrm{data}) L_2(\theta|\mathrm{data})$$

Then we simply maximize $$L_1(\beta|\mathrm{data})$$.

Profile Likelihood

If we can express $$\theta$$ as a function of $$\beta$$ then we replace $$\theta$$ with the corresponding function.

Say, $$\theta = g(\beta)$$. Then, we maximize:

$$L(\beta, g(\beta)|\mathrm{data})$$

Marginal Likelihood

We integrate out $$\theta$$ from the likelihood equation by exploiting the fact that we can identify the probability distribution of $$\theta$$ conditional on $$\beta$$.

• Note that the last definition here is an Integrated (or Bayesian) Likelihood, not a Marginal Likelihood.
– ars
Jul 31, 2010 at 0:09
• Is this correct in the RHS for partial likelihood: "L2(θ|theta)"? Jul 31, 2010 at 0:13
• @ars, would you please edit the answer and provide the definition of Marginal Likelihood then? Oct 3, 2016 at 17:06
• hmm, the standard way of defining likelihood is the probability of the DATA GIVEN the PARAMETERS, that is, the inverse conditioning of what's written here Jul 6, 2020 at 21:40
• @AndreasSteimer every reference I have come across has always defined the likelihood as a function of the parameters given fixed data. The opposite situation where probabilities are returned as a function of data given fixed parameters would simply be a probability distribution or density. Nov 8, 2021 at 6:21

All three are used when dealing with nuisance parameters in the completely specified likelihood function.

The marginal likelihood is the primary method to eliminate nuisance parameters in theory. It's a true likelihood function (i.e. it's proportional to the (marginal) probability of the observed data).

The partial likelihood is not a true likelihood in general. However, in some cases it can be treated as a likelihood for asymptotic inference. For example in Cox proportional hazards models, where it originated, we're interested in the observed rankings in the data (T1 > T2 > ..) without specifying the baseline hazard. Efron showed that the partial likelihood loses little to no information for a variety of hazard functions.

The profile likelihood is convenient when we have a multidimensional likelihood function and a single parameter of interest. It's specified by replacing the nuisance S by its MLE at each fixed T (the parameter of interest), i.e. L(T) = L(T, S(T)). This can work well in practice, though there is a potential bias in the MLE obtained in this way; the marginal likelihood corrects for this bias.