We define likelihood of parameters $\theta$ given observations $x$ (assuming $x$ is sampled according to density $f$) as: $$\mathcal{L}(\theta | x)=f_\theta(x). $$ Is it correct to speak about "[l]ikelihood of the data" (The Elements of Statistical Learning before equation (2.35)), or should we only speak about the likelihood of parameters?

I also get such remark during oral presentation, but I'm still confused to know if this misnomer is accepted.

In addition, the first equation comes from the English Wikipedia, but in the French ones it is written: $$\mathcal{L}(x | \theta)=f(x;\theta).$$ I think this notation $\mathcal{L}(x | \theta)$ is incorrect, but is it accepted somehow? Are there some authoritative notation rules for likelihood, or each author picks as he wants?


I don't know about "authoritative sources", but I would say the notation from French Wikipedia is strange, and I have never seen it. Also, I wold replace the one you attribute to English Wikipedia with $$ \mathcal{L}(\theta | x)\propto f_\theta(x). $$ (where we assume the constant of proportionality is positive.) For proportionality see What does "likelihood is only defined up to a multiplicative constant of proportionality" mean in practice?.

So likelihood is a measure of support for a model, and as long as we look at a parameterized family of models, that will be a function of the parameters. So say likelihood for the parameter $\theta$ given the data $x$. Likelihood for the data doesn't give much meaning for me.


You don't speak of likelihood of the data. Of the four possible ways to mix likelihood/probability and data/parameters, you speak of:


  • likelihood of the parameters (given the data)


  • probability or probability density of the data (given the parameters)

In a Bayesian probability setting, where the parameters also follow a probability distribution, you may speak as well of (posterior/prior) probability of parameters.

The difference depends on which part of the equation is considered fixed.

Likelihood is considered a function of the parameters (ie theta) given a particular observation $x$ $$\mathcal{L}(\theta|x)$$ The vertical bar $|$ is used to say that $x$ is a fixed parameter and $\theta$ is variable (but of course, you can encounter different $x$ and it is only fixed for the particular case).

The likelihood relates to inverse probability, which contrasts with the typical concept of probability that gives:

  • The probability of observations given some set of parameters.

This typical concept of probability is often a straightforward problem. The problem of inverse probability is to express:

  • The probability (or something related) of the set of parameters given some observation.

Which is not straightforward, or even an impossible (ill-defined) problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.