My understanding of the Probabilty Density Function is that they evaluate to 0 at a particular point . So if we have some i.i.d points $x_{n}$ from a Normal distribution and we write :
\begin{equation} N(x|\mu,\sigma^{2}) = \frac{\exp\left(\frac{-(x-\mu)^{2}}{2\sigma^{2}}\right)}{\sqrt{2\pi\sigma^{2}}} \end{equation} Then $P(x_{n}) = 0$.
Now i am reading this book "Pattern Recogniton and Machine Learning -Bishop ". In the book the author talks of a data set of observations $x=(x_{1},x_{2},...,x_{n})$ which are i.i.d .He then says the we can theresfore write the $\textbf{Probability}$ of the data set , given $\mu$ and $\sigma^{2}$ in the form :

\begin{equation} p(X|\mu,\sigma^{2}) = \prod_{n=1}^{N} N(x_{n}|\mu,\sigma^{2}) \end{equation}

$\textbf{My Question : }$
If i expand the product $\prod$ , the individual terms dont look like a probability to me but indeed the value of the function ( Normal Density function) evaluated at points $x_{i}$. So how can we say that the expression : \begin{equation} \prod_{n=1}^{N} N(x_{n}|\mu,\sigma^{2}) \end{equation} represents the Probability of the data set/joint P.D.F of $x_{i}, i \in [1,n]$

  • 2
    $\begingroup$ Especially in Bayesian discussions, I've seen notation such as $p(X|\mu,\sigma^{2})$ (with $p$ instead of $f$) used for density and likelihood functions. For continuous distributions there is no implication that $p$ stands directly for a 'probability'. $\endgroup$ – BruceET Apr 21 '19 at 2:14
  • $\begingroup$ PDFs are the probability density function. They do not represent probability but instead the density of probability at any point. Thus they do not evaluate to 0. For instance, the only points at which a normal PDF evaluates to 0 are plus\minus infinity. Similarly for the joint distribution, the expression does not gives the probability, but the density of probability at any sample value. $\endgroup$ – Sanket Agrawal Apr 21 '19 at 5:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.