# If Likelihood is not a PDF then why is the PDF of Multivariate Normal equivalent to the likelihood of I.I.D. Normals?

I am understanding why likelihoods are not PDFs using links such as What is the reason that a likelihood function is not a pdf. However I am getting more confused.

For instance, the likelihood of I.I.D. Normals which can be seen as $$\prod^N \mathcal N(x_n|\mu,\sigma^2) =\frac{1}{Z^N}\prod \frac{1}{2} (X-\mu)^\top (\sigma I)^{-1}(X-\mu)$$

This is equivalent with the PDF of an MVN $$N(\mu,\sigma^2 I)$$.

This may also be the reason why maximizing the likelihood or the MVN PDF of the data with Covariance matrix $$\sigma^2 I$$ yields the same set of solution (particularly for linear regression).

Please help me clarify this. Is a reason why the likelihood is not considered a PDF is because of context? (The likelihood is meant to be a function that should be maximized with respect to the parameters)

The equivalence between the likelihood and the MVN above is just a coincidence? In fact I think any likelihood can be integrated to $$1$$ with a suitable normalizing constant unless maybe the likelihood itself does not converge under the integral.

• Your $\prod^N \mathcal N(x_n|\mu,\sigma^2)$ is a possible pdf for $\mathbf X$ since it is non-negative and integrated over all possible values of $\mathbf X$ is $1$. It could not be a pdf for $(\mu, \sigma^2)$ as its integral over all possible values of $\mathbf X$ is not $1$. Jun 20, 2023 at 8:17
• @Henry did you also mean that $P(\mu,\sigma^2|\mathbf X)$ can not be considered as a pdf since it does not integrate to $1$ over all possible values of $\mathbf X$? Jun 20, 2023 at 8:22
• $P(\mu,\sigma^2\mid \mathbf X)$ is the wrong notation here for a likelihood. I am saying $\Lambda(\mu,\sigma^2 \mid \mathbf X)$ is not a pdf for $(\mu,\sigma^2)$ and it does not integrate to $1$ over all possible values of $(\mu, \sigma^2)$ Jun 20, 2023 at 8:39
• @Henry can I safely conclude that the PDF of the MVN evaluated at $\mathbf X$ and treating $\mu,\sigma$ as the argument variables makes it a likelihood? Jun 20, 2023 at 13:06
• Yes - that is essentially what the first sentence of Sextus Empiricus's answer says. Jun 20, 2023 at 13:37

The likelihood inverts the relationship between random data and the parameters of the distribution from which the data are sampled. Then due to the symmetry of the expression $$(x-\mu)$$ the likelihood function will resemble the pdf.

$$f(x,\mu) = \frac{1}{\sqrt{2 \pi}} e^{-\frac{1}{2} (x-\mu)^2 } = \frac{1}{\sqrt{2 \pi}} e^{-\frac{1}{2} (x^2-2x\mu +\mu^2) }$$

This function will have the same shape when you switch $$\mu$$ and $$x$$ in the equations. (But note that having the shape of a pdf will not make it the same as a pdf*.)

For many other functions the likelihood function is not the same shape as the pdf. An example is the following pdf of $$\hat{\theta}$$ given the distribution parameter $$\theta$$

$${\hat\theta \sim \mathcal{N}(\mu=\theta, \sigma^2=1+\theta^2/3)}$$

or

$$f(\hat\theta, \theta ) = \frac{1}{\sqrt{2 \pi (1+\theta^2/3)}} \exp \left[ \frac{-(\theta-\hat\theta)^2}{2(1+\theta^2/3)} \right]$$

Imagine this probability density function $$f(\hat \theta , \theta)$$ plotted as a function of $$\theta$$ and $$\hat \theta$$. This is done as a surface plot and a contour plot in the image below.

You can see the likelihood function as a particular slice of the function for a fixed value of the observed $$\hat{\theta}$$ (also projected on the right), and it does not have the same shape as a normal distribution.

Other parts of the images, like the red and green curves and dots, are details that relate to the question The basic logic of constructing a confidence interval . The image shows how the boundaries of a confidence interval relate to particular points of the pdf/cdf at different values of the parameter, and how it is different from a likelihood interval.

• Then the equivalence is indeed a coincidence due to the symmetry? The likelihood assumes a fixed value for the $x_i$ and it is a function of the parameters. While the MVN PDF is the density of $x_i$, (a random variable) given fixed set of paremeters? In this sense, usage context also plays a part here. Sorry for the long comment but I need to have deeper understanding with this. Jun 20, 2023 at 8:00
• @user1176663 the likelihood is a function of probabilities, but that does not yet make it a probability distribution. Jun 20, 2023 at 8:09
• A similar difference occurs with the fiducial distribution which is a function of probability densities but does not behave like a probability density distribution. stats.stackexchange.com/a/592783 Jun 20, 2023 at 8:11