I keep seeing density functions that don't explicitly arise from conditioning written with the conditional sign: For example for the density of the Gaussian $N(\mu,\sigma)$ why write: $$ f(x| \mu, \sigma)=\frac{1}{\sqrt{2\pi \sigma^2}}\exp{-\frac{(x-\mu)^2}{2\sigma^2}}$$

instead of

$$ f(x)=\frac{1}{\sqrt{2\pi \sigma^2}}\exp{-\frac{(x-\mu)^2}{2\sigma^2}}$$

Is this done purely to be explicit as to what the parameter values are or(what I'm hoping for) is there some meaning related to conditional probability?

  • In a Bayesian context, the parameters are random variables, so in that context the density is actually the conditional density of $X \mid (\mu, \sigma)$. In that setting, the notation is very natural.
  • Outside of a Bayesian context, it is just a way to make it clear that the density depends (here I am using this word colloquially, not probabilistically) on the parameters. Some people use $f_{\mu, \sigma}(x)$ or $f(x; \mu, \sigma)$ to the same effect.
  • This latter point can be important in the context of likelihood functions. A likelihood function is a function of the parameters $\theta$, given some data $x$. The likelihood is sometimes written as $L(\theta \mid x)$ or $L(\theta ; x)$, or sometimes as $L(\theta)$ when the data $x$ is understood to be given. What is confusing is that in the case of a continuous distribution, the likelihood function is defined as the value of the density corresponding to the parameter $\theta$, evaluated at the data $x$, i.e. $L(\theta; x) := f_\theta(x)$. Writing $L(\theta; x) = f(x)$ would be confusing, since the left-hand side is a function of $\theta$, while the right-hand side ostensibly does not appear to depend on $\theta$. While I prefer writing $L(\theta; x) := f_\theta(x)$, some might write $L(\theta; x) := f(x \mid \theta)$.
  • I have not really seen much consistency in notation across different authors, although someone more well-read than I can correct me if I am wrong.
  • $\begingroup$ if you find point 3 confusing, that simply means that you don't understand likelihood; that kind of inversion is the whole point of having a likelihood function $\endgroup$
    – carlo
    Oct 28 '20 at 19:21

This notation is used often in MLE context to differentiate it from likelihood function and the estimation of the parameters conditional on data.

In MLE you do something like this: $$\hat\mu,\hat\sigma|X= \underset{\mu,\sigma}{\operatorname{argmax}} \mathcal L(X|\mu,\sigma)$$ $$\mathcal L(X|\mu,\sigma)=\prod_i f(x_i\in X|\mu,\sigma) $$

So, this notation emphasizes that you use the PDF $f(.)$ of the data set conditional on a candidate set of parameters to obtain the likelihood function $\mathcal L$. Then you pick the set that maximizes the likelihood as your solution $\hat\mu,\hat\sigma$. Thus, the solution is truly conditional on the data set $X$, while the likelihood is conditional on the candidate parameter set $\mu,\sigma$. That's why this notation is good for didactic purpose to show how the conditions sort of "flip" on left- and right-hand side.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.