6
$\begingroup$

In this discussion by comments Is the exact value of any likelihood meaningless?, it was suggested (firmly!) that likelihoods and probabilities calculated from continuous data not only have units, but those units are the reciprocal of units of the data. In contrast, likelihoods and probabilities from discrete data have no units. Is that correct?

I thought that probabilities and likelihoods were dimensionless and had not thought about such a stark contrast coming from the continuity of observed values.

Improve this question
$\endgroup$
5
  • 4
    $\begingroup$ Likelihoods, in the case of continuous models, comes from densities, not from probabilities ... and densities clearly have units, namely, probability per unit of the variable that owns the density. That way, when integrating the density we get the units of probability (that is, 1) $\endgroup$
    – kjetil b halvorsen ♦
    Commented Jun 15, 2022 at 22:17
  • 1
    $\begingroup$ If you adopt the perspective used in my analysis of likelihoods at stats.stackexchange.com/a/397166/919, there won't be any issue: all likelihoods are probabilities, but for our calculations in the continuous case we ignore a factor that does have units (namely, the differential element). $\endgroup$
    – whuber ♦
    Commented Jun 15, 2022 at 22:46
  • $\begingroup$ @kjetilbhalvorsen Are the units the units of the data values, or are they the units of the model parameter(s)? Are the units the same for probabilities and likelihoods? $\endgroup$
    – Michael Lew
    Commented Jun 15, 2022 at 22:56
  • 1
    $\begingroup$ The units come from the differential term in the probability element--the $\mathrm{d}x$ etc. $\endgroup$
    – whuber ♦
    Commented Jun 15, 2022 at 23:27
  • $\begingroup$ If probabilities have no units and densities have units like the reciprocal of the units of the underlying random variable then with two continuous random variables $f_{\Theta\mid X=x}(\theta)=\dfrac{f_{X\mid \Theta=\theta}(x) f_{\Theta}(\theta)}{f_X(x)}$ suggests $f_{X\mid \Theta=\theta}(x)$ being used as the likelihood has units like $f_X(x)$ the reciprocal of the units of $X$. But if you use the proportionality property for $f_{\Theta\mid X=x}(\theta)=\dfrac{L_{x}(\theta) f_{\Theta}(\theta)}{\int L_{x}(\phi) f_{\Theta}(\phi) \, d \phi}$ then $L_{x}(\theta)$ can have any units $\endgroup$
    – Henry
    Commented Jun 16, 2022 at 8:52

1 Answer 1

Reset to default
7
$\begingroup$

Probabilities (also called "probability masses") are unitless, but probability densities have units of 1/(units of the variable).

Let's say we have a probability density $p\left(x\right)$ of some variable $x$. If we integrate this density over some range in $x$, we obtain the probability that $x$ falls in this range: $$ P\left(a < x < b\right) = \int_a^b p\left(x\right) dx . $$ The quantity $P\left(a < x < b\right)$ is, of course, a probability mass, and must therefore be a unitless real number between 0 and 1. But in order for $P\left(a < x < b\right)$ to be unitless, we can see from the above integral that $p\left(x\right)$ must have units of 1/(units of $x$). The quantity $p\left(x\right)$ is the amount of probability per unit of $x$, which is why the term "probability density" is applied to it in the first place.

likelihoods and probabilities calculated from continuous data not only have units, but those units are the reciprocal of units of the data

"Probabilities calculated from continuous data" only have units of 1/(units of the data) if they are probability densities of the data. The likelihood is the probability density of the data (assuming the data to be made up of continuous variables - for discrete data, the likelihood is a unitless probability mass), given a choice of model. The likelihood is usually given as a function of the model parameters, $\theta$: $$ L\left(\theta\right) := p\left(D|\theta\right) , $$ where $D$ is the data. As such, the likelihood has units of 1/(units of $D$), and its integral over all possible values of $D$ must come to 1. However, very importantly, $L\left(\theta\right)$ is not a probability density of $\theta$, and the integral of $L\left(\theta\right)$ over all values of $\theta$ does not have to come to 1. In fact, that integral might not even yield a unitless quantity at all: it will have units of (units of $\theta$)/(units of $D$).

This often leads to considerable confusion, because students are rightly taught that $L\left(\theta\right)$ is not a probability density in $\theta$, but then incorrectly conclude that it's not a probability density at all. In fact, it's the probability density of the data (given the model).

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Thanks apdnu, that makes it a bit more clear. However, then next step is for me to understand what I have to do with my new knowledge. Have you looked at Whuber's comment under the question? It suggests that the units of likelihoods do not require me to do anything at all and so I can continue to act as if they were unitless... $\endgroup$
    – Michael Lew
    Commented Jun 15, 2022 at 23:10
  • $\begingroup$ @MichaelLew: This answer might help explain units a bit. $\endgroup$
    – Nat
    Commented Jun 16, 2022 at 9:54
  • 1
    $\begingroup$ Could you give example(s) where the units of the likelihood are important? $\endgroup$
    – dipetkov
    Commented Jun 17, 2022 at 21:42
  • 1
    $\begingroup$ Let's say you're trying to explain observed data with a mixture model, in which each data point can be explained either by a physical model (presumably the model you're really interested in) or an outlier model (which describes everything you're not interested in). In order to write down the full model, you need to properly normalize both component models. Those normalization factors will have units. $\endgroup$
    – apdnu
    Commented Jun 18, 2022 at 12:17
  • 1
    $\begingroup$ Thanks a lot for this answer and for the example. I struggled a lot with the original "meaningless" statement, which seemed wrong and glib but I couldn't come up with a convincing counter-example where all likelihood terms mattered, incl. those that only depend on the data. $\endgroup$
    – dipetkov
    Commented Jun 28, 2022 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.