Correct way to speak and write about likelihoods

Question

We define likelihood of parameters $\theta$ given observations $x$ (assuming $x$ is sampled according to density $f$) as: $$\mathcal{L}(\theta | x)=f_\theta(x). $$ Is it correct to speak about "[l]ikelihood of the data" (The Elements of Statistical Learning before equation (2.35)), or should we only speak about the likelihood of parameters?

I also get such remark during oral presentation, but I'm still confused to know if this misnomer is accepted.

---

In addition, the first equation comes from the English Wikipedia, but in the French ones it is written: $$\mathcal{L}(x | \theta)=f(x;\theta).$$ I think this notation $\mathcal{L}(x | \theta)$ is incorrect, but is it accepted somehow? Are there some authoritative notation rules for likelihood, or each author picks as he wants?

Answer 1

I don't know about "authoritative sources", but I would say the notation from French Wikipedia is strange, and I have never seen it. Also, I wold replace the one you attribute to English Wikipedia with $$ \mathcal{L}(\theta | x)\propto f_\theta(x). $$ (where we assume the constant of proportionality is positive.) For proportionality see What does "likelihood is only defined up to a multiplicative constant of proportionality" mean in practice?.

So likelihood is a measure of support for a model, and as long as we look at a parameterized family of models, that will be a function of the parameters. So say likelihood for the parameter $\theta$ given the data $x$. Likelihood for the data doesn't give much meaning for me.

Answer 2

You don't speak of likelihood of the data. Of the four possible ways to mix likelihood/probability and data/parameters, you speak of:

either

• likelihood of the parameters (given the data)

or

• probability or probability density of the data (given the parameters)

In a Bayesian probability setting, where the parameters also follow a probability distribution, you may speak as well of (posterior/prior) probability of parameters.

---

The difference depends on which part of the equation is considered fixed.

Likelihood is considered a function of the parameters (ie theta) given a particular observation $x$ $$\mathcal{L}(\theta|x)$$ The vertical bar $|$ is used to say that $x$ is a fixed parameter and $\theta$ is variable (but of course, you can encounter different $x$ and it is only fixed for the particular case).

The likelihood relates to inverse probability, which contrasts with the typical concept of probability that gives:

• The probability of observations given some set of parameters.

This typical concept of probability is often a straightforward problem. The problem of inverse probability is to express:

• The probability (or something related) of the set of parameters given some observation.

Which is not straightforward, or even an impossible (ill-defined) problem.