I've been looking through some literature on probability theory, and I have noticed that some authors seem to use the term "distribution" very loosely, and more than once when referring to probability densities. An example from the wikipedia article on marginal distributions:

Similarly for continuous random variables, the marginal probability density function can be written as $p_X(x)$. This is $$p_X(x) = \int p_{X|Y}(x|y)p_Y(y)dy $$ where $p_{X,Y}(x,y)$ gives the joint distribution of X and Y, while $p_{X|Y}(x|y)$ gives the conditional distribution for X given Y.

Again, from the article on Posterior Probability:

The posterior probability distribution of one random variable given the value of another can be calculated with Bayes' theorem by multiplying the prior probability distribution by the likelihood function, and then dividing by the normalizing constant, as follows: enter image description here

gives the posterior probability density function for a random variable X given the data Y=y, where ...

Now a probability density determines a distribution, but they are not interchangable. The reason I ask is I am reading some algorithms and proofs for MCMCs and it is often very unclear whether the computations are done via densities or probabilites, and looking at different sources is often more confusing than helping, since the terminology seems to differ. Am I missing something, or is this inconsistant usage of the terms?

  • 3
    $\begingroup$ The first quotation isn't "loose" at all. Notice its use of the word "gives," which perhaps you are over-interpreting as "equals." $\endgroup$ – whuber Mar 30 '18 at 14:12
  • 2
    $\begingroup$ Check out the last paragraph of the opening to the Probability Density Function article--it also describes the inconsistency among authors and may be illuminating. $\endgroup$ – Steve S Mar 30 '18 at 14:31

The distribution of a random variable is a measure, defined on the space where the random variable takes its values. It encodes the probability of every event you are considering. So, when you have a real valued random variable, its distribution is a measure on $\mathbb{R}$. Sometimes this measure can be described by a probability density function, but not always.

Since the the density function fully describes the distribution, you can refer to "calculating the density function" as "calculating the distribution". So I see no problem with:

The posterior probability distribution of one random variable given the value of another can be calculated...

where $p_{X,Y}(x,y)$ gives the joint distribution of X and Y...

Keep in mind that you are only interested in knowing that function because it represents a distribution, which is what you are really interested in.

What I do find somewhat far fetched is:

by multiplying the prior probability distribution by the likelihood function...

Because here the object you really are multiplying is the density function. On the other hand, it's a little bit like this: Let's say you are searching through a photo album and someone comes along and asks what you are searching for. You respond:

I want to see if I can find Alice in here

It's clear that you mean "to find a photograph in which Alice is depicted" and you are not expecting to find the real person Alice in that album. The same way "multiplying the prior probability distribution" intends to refer to "multiplying the density function that describes the prior probability distribution" in that case.

Something to add: The reason why it's clear in the above example that a photograph of Alice is meant rather than Alice itself is the availability of additional knowledge. When writing about mathematics, whenever commensurate, it's better to do it in a way that doesn't rely on additional knowledge on the reader's side.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.