4
$\begingroup$

Claim 1: For continuous random variable, $P(X=x)=0$, where $x$ is a particular number.

Claim 2: When we use maximum likelihood estimation, we plug-in mean, standard deviation and data point $x$ into the formula like this:

$$L\color{red}{(\mu,\sigma|x)}=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-(x-\mu)^2/2\sigma^2}$$

Then we will get $L(\mu, \sigma|x)$ equal to a non-zero value. However, on the right-hand side of the above formula, that's exactly the pdf of normal distribution!

Is there a contradiction between Claim 1 and Claim 2? If not, am I missing something, and why the two claims can be valid simultaneously?

$\endgroup$
2

2 Answers 2

10
$\begingroup$

There isn't any contradiction here. Your confusion is a very common one when dealing with the likelihood function for a continuous random variable.

Let's take a step back and start from a discrete random variable (r.v.). For discrete r.v., $P(X=x)>0$ for finitely or infinitely many $x$, depending on the r.v. involved, and since the likelihood is the product of probabilities, it can actually be interpreted as

the probability of observing the sample actually observed if data were generated under that model with the same parameter value.

Now, for a continuous r.v., $P(X=x)=0$ for all $x$, thus we cannot use the above interpretation/construction. However, if the continuous r.v. has a density, then the likelihood function is defined as the product of densities. This time the likelihood function tells us

the degree of likelihood to observe the sample we actually observed under the given model with the fixed parameter value.

Apart from this slight variation in the interpretation, in both discrete and continuous r.v. the likelihood function is used in a relative way and not in an absolute way.

That is, knowing that the likelihood of $\theta_1$ is $10^{-3}$ is not directly that useful. However, if we also know that the likelihood of $\theta_2$ is $10^{-2}$, then we can say that the likelihood that data are generated under $\theta=\theta_2$ is 10 times larger than the likelihood that they are generated under $\theta=\theta_1$.

Interpretation aside, mathematically we see the likelihood as a function of $\theta$, by holding the data fixed. Indeed, the notation $L(\theta)$, which is often used to denote the likelihood function, suppresses the dependence on $x$ for this reason. On the other hand, we see the density function $f(x;\theta)$ as a function of $x$, holding $\theta$ fixed.

$\endgroup$
1
  • 2
    $\begingroup$ if the continuous r.v. has a density, then the likelihood function is defined as the product of densities. This articulates it properly. $\endgroup$ Jan 2, 2023 at 11:27
0
$\begingroup$

The PMF ($\Pr[X=x]$) of a CRV is always zero because points are infinitesimally small, no matter how likely they are to be chosen. The PDF is designed for continuous random variables, so it doesn't have that problem. It uses calculus (the instantaneous rate of change of the CDF). The likelihood of a CRV is defined on the PDF (because the PMF of a CRV is "broken"). There's no conflict here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.