According to the Wikipedia article Likelihood function, the likelihood function is defined as:

$$ \mathcal{L}(\theta|x)=P(x|\theta), $$

with parameters $\theta$ and observed data $x$. This equals $p(x|\theta)$ or $p_\theta(x)$ depending on notation and whether $\theta$ is treated as random variable or fixed value.

The notation $\mathcal{L}(\theta|x)$ seems like an unnecessary abstraction to me. Is there any benefit to using $\mathcal{L}(\theta|x)$, or could one equivalently use $P(x|\theta)$? Why was $\mathcal{L}(\theta|x)$ introduced?

  • 6
    $\begingroup$ In this context, it reminds us that the likelihood function is a function of $\theta$ with the data $x$ fixed. On the other hand, the joint distribution is a function of the data $x$ given $\theta$. $\endgroup$
    – knrumsey
    Commented Jun 11, 2017 at 22:25
  • 1
    $\begingroup$ @BigAgnes Thanks. Aren't observed variables fixed by definition, though? I'm also confused why we can call $p(x|\theta)$ a joint distribution. Isn't it a scalar since both $x$ and $\theta$ are fixed (assuming a Frequentist approach where $\theta$ is not a random variable). $\endgroup$
    – danijar
    Commented Jun 11, 2017 at 22:34
  • 1
    $\begingroup$ Closely related: stats.stackexchange.com/questions/224037/… $\endgroup$
    – Tim
    Commented Jun 12, 2017 at 7:50

6 Answers 6


Likelihood is a function of $\theta$, given $x$, while $P$ is a function of $x$, given $\theta$.

Roughly like so (excuse the quick effort in MS paint):

"3D" plot showing a set of densities running left to right and likelihoods running front to back

In this sketch we have a single $x$ as our observation. Densities (functions of $x$ at some $\theta$) are in black running left to right and the likelihood functions (functions of $\theta$ at some $x$) are in red, running front to back (or rather back to front, since the $\theta$ axis comes 'forward' and somewhat to the left). The red curves are what you get when you 'slice' across the set of black densities, evaluating each at a given $x$. When we have some observation, it will 'pick out' a single red curve at $x=x_\text{obs}$.

  • The likelihood function is not a density (or pmf). It is defined in terms of density but it's a different one at every point. It doesn't integrate (/sum) to 1. It needn't even be normalizable.

  • Indeed, $\mathcal L$ may be continuous while $P$ is discrete (e.g. likelihood for a binomial parameter) or vice-versa (e.g. likelihood for an Erlang distribution with unit rate parameter but unspecified shape)

Imagine a bivariate function of a single potential observation $x$ (say a Poisson count) and a single parameter (e.g. $\lambda$) -- in this example discrete in $x$ and continuous in $\lambda$ -- then when you slice that bivariate function of $(x,\lambda)$ one way you get $p_\lambda(x)$ (each slice gives a different pmf) and when you slice it the other way you get $\mathcal L_x(\lambda)$ (each a different continuous likelihood function).

(That bivariate function simply expresses the way $x$ and $\lambda$ are related via your model)

Conversely, with a discrete $\theta$ and a continuous $x$ the likelihood is discrete and the density continuous.

As soon as you specify $x$, you identify a particular $\mathcal L$, that we call the likelihood function of that sample. It tells you about $\theta$ for that sample -- in particular what values had more or less likelihood of giving that sample.

Likelihood is a function that tell you about the relative chance that this value of $\theta$ could produce your data (in that ratios of likelihoods can be thought of as ratios of probabilities of being in the interval from $x$ to $x+dx$), when comparing it to other values for $\theta$.

  • 1
    $\begingroup$ It's not a density. For any given $\theta$ its value is equal to that of a density evaluated at a specific $x$, but it's equal to a different density at every $\theta$. Imagine you took every possible value of $\theta$ (imagine for the moment a discrete $\theta$ but continuous $p$) and for each one, you drew the probability density, $p$). Then at the specific sample value ($x$), you slice orthogonally across all those different densities. That slice is a likelihood function --- and it is not itself a density. $\endgroup$
    – Glen_b
    Commented Nov 12, 2019 at 22:45
  • 1
    $\begingroup$ The second thing. For each specific value of $\theta$ and a given $x$, $L$ is equal to the value of the density evaluated at that $x$, given that $\theta$. but the density changes with $\theta$, so $L$ is equal to the value of a different density (each evaluated at $x$) at every point on $L$. $\endgroup$
    – Glen_b
    Commented Nov 12, 2019 at 23:27

According to the Bayesian theory, $f(\theta \mid x_1,\ldots,x_n) = \frac{f(x_1,\ldots,x_n|\theta) \cdot f(\theta)}{f(x_1,\ldots,x_n)}$ holds, that is $\text{posterior} = \frac{\text{likelihood} \cdot \text{prior}}{\textrm{evidence}}$.

Notice that the maximum likelihood estimate omits the prior beliefs(or defaults it to zero-mean Gaussian and count on it as the L2 regularization or weight decay) and treats the evidence as constant(when calculating the partial derivative with respect to $\theta$).

It tries to maximize the likelihood by adjusting $\theta$ and just treating $f(\theta\mid x_1,\ldots ,x_n)$ equal to $f(x_1,\ldots,x_n\mid \theta)$ which we can easily get(usually the loss) and keep the likelihood as $\mathcal{L}(\theta\mid \mathbf x)$. The true probability $\frac{f(x_1,\ldots,x_n|\theta) \ldots f(\theta)}{f(x_1,\ldots,x_n)}$ can hardly be worked out because the evidence(the denominator), $\int_{\theta} f(x_1, \ldots,x_n, \theta)d\theta$, is intractable.

Hope this helps.


I agree with @Big Agnes. Here is what my professor taught in class: One way is to think of likelihood function $L(\theta | \mathbf{x})$ as a random function which depends on the data. Different data gives us different likelihood functions. So you may say conditioning on data. Given a realization of data, we want to find a $\hat{\theta}$ such that $L(\theta | \mathbf{x})$ is maximized or you can say $\hat{\theta}$ is most consistent with data. This is same to say we maximize "observed probability" $P (\mathbf{x} | \theta)$. We use $P(\mathbf{x} | \theta)$ to do calculation but it is different from $P(\mathbf{X} | \theta)$. Small $\mathbf{x}$ stands for observed values, while $\mathbf{X}$ stands for random variable. If you know $\theta$, then $P(\mathbf{x} | \theta)$ is the probability/ density of observing $\mathbf{x}$.

  • $\begingroup$ Thanks. Could I equivalently use $P(x|\theta)$ (with lowercase x) instead of $\mathcal{L}(\theta|x)$? When we write ${max}_\theta{P(x|\theta)}$ it should be clear that $x$ is fixed and we're trying to find the most consistent $\theta$. Or does $\mathcal{L}(\theta|x)$ refer to something more abstract that has a different implementation in some situations? $\endgroup$
    – danijar
    Commented Jun 11, 2017 at 22:52
  • $\begingroup$ Also, could you elaborate why $\mathcal{L}(\theta|x)$ is a random function? It seems like it should be deterministic since both $x$ and $\theta$ are fixed (unless we give $\theta$ a Bayesian treatment and consider it a random variable). $\endgroup$
    – danijar
    Commented Jun 11, 2017 at 22:54
  • $\begingroup$ It is better to use $L(\theta | \mathbf{x})$ (actually likelihood is defined in such way), because it is a function of $\theta$ rather than $\mathbf{x}$. I don't know if $L(\theta | \mathbf{x})$ refers to something abstract. As for random function argument, it is just a way of thinking of likelihood function. The true $\theta$ is fixed, but we don't know it. That's why we estimate it. You plug in your observations into $L(\theta | \mathbf{x})$, and different data gives you different functions. So likelihood function depends on your observation, so it is like a function of random variables. $\endgroup$
    – jwyao
    Commented Jun 11, 2017 at 23:09
  • $\begingroup$ $L(\theta | \mathbf{x})$ looks like a posterior distribution but in fact, it isn't. There is no assumption on the (prior) distribution $\pi (\theta)$ of $\theta$. $\endgroup$
    – jwyao
    Commented Jun 11, 2017 at 23:11
  • $\begingroup$ So one could write $\mathcal{L}_x(\theta)$ to express this more clearly? (I know we shouldn't write this in practice since it's not common notation.) $\endgroup$
    – danijar
    Commented Jun 11, 2017 at 23:17

I think the other answers given by jwyao and Glen_b are quite good. I just wanted to add a very simple example which is too long for a comment.

Consider one observation $X$ from a Bernoulli distribution with probability of success $\theta$. With $\theta$ fixed (known or unknown), the distribution of $X$ is given by $p(X|\theta)$.

$$P(x|\theta) = \theta^x(1-\theta)^{1-x}$$

In other words, we know that $P(X=1) = 1 - P(X=0) = \theta$.

Alternatively, we could look treat the observation as fixed and view this as a function of $\theta$.

$$L(\theta | x) = \theta^x(1-\theta)^{1-x}$$

For example, in a maximum likelihood setting, we seek to find $\theta$ which maximizes the likelihood as a function of $\theta$. For example, if we observe $X = 1$, then the likelihood becomes

$$L(\theta | x) = \begin{cases} \theta, & 0 \leq \theta \leq 1 \\ 0, & \text{else} \end{cases}$$

and we see that the MLE would be $\hat\theta = 1$.

Not sure that I've really added any value to the discussion, but I just wanted to give a simple example of the different ways of viewing the same function.


When we write $f(z|\omega)$ it means a function of $z$ for parameters $\omega$. The density $P(x|\theta)$ is a function of data $x$ for parameters $\theta$, e.g. $\int_X P(x|\theta)dx=1$.

When we define likelihood function $\mathcal{L}(\theta|x)$ it means a function of parameters $\theta$ given the dataset $x$. For instance, $\int_\Theta\mathcal{L}(\theta|x)d\theta\ne 1$.

It's important distinction because the optimization problem is over parameters $\theta$ and not the dataset $x$:$$\min_\Theta \mathcal{L}(\theta|x)$$ In fact, we have only one dataset in this case. Therefore, despite we define likelihood with equality $\mathcal{L}(\theta|x)=P(x|\theta)$, it is important to denote it as a function of parameters $\theta$.


L(θ) = P(y|x;θ) L(θ): likelihood of θ where θ is not a random variable it is a parameter P(y|x;θ): Probability of observing y given data x when parameter θ is used.

Given data in format of x = [x1, x2, x3, ..., xn] => y (n feature input and y as our predefined output in our training data)

So, maximizing L(θ) helps us get θ parameter which will increase chances of observing y when x given. (Maximizing probability of getting result h(θx) = y)

  • 1
    $\begingroup$ What does this add to the already existing answers? $\endgroup$ Commented Aug 8, 2023 at 0:34
  • $\begingroup$ I am sorry, but I cannot make any sense out of this answer. $\endgroup$
    – jbowman
    Commented Aug 8, 2023 at 1:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.