I am reading through Christopher Bishop's Pattern Recognition and Machine Learning book and came across this part that I did not quite understand. enter image description here As I understand it, this is claiming that the probability of a set of data is equal to the product of the p.d.f. of the normal distribution of the datapoints. However, the p.d.f. can be greater than one, especially if $\sigma$ is small. So if it happens that I sample a large amount of my data near the mean, isn't it possible to return a value greater than 1 for the probability of the dataset? I feel I have a misunderstanding somewhere since this shouldn't be possible.


1 Answer 1


You're quite correct; as many posts on site reiterate, density is not probability, and densities can certainly exceed 1.

However density is proportional to a probability. e.g. in the univariate case the probability of being in the interval $(x,x+\delta )$ would (as $\delta$ becomes very small) approach $p(x)\delta$ (this may be put more formally but my aim here is not to be formal).

Bishop is not correct to call that conditional density "the probability of the data set" though we might reasonably assume that's loose terminology rather than lack of understanding.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.