# How does the marginal distribution become the prior distribution?

Bayes' Theorem for densities/pmf's states that, given, say, two univariate random variables $x,z$ we have

$$p(z\mid x) = \frac {p(x\mid z)\cdot p(z)}{p(x)}$$

This is part of the core of the probability theory most widely used nowadays.

But in order to use this theoretical result for Bayesian statistical inference, two basic mappings are apparently made:

• First, $p(x\mid z)$, goes from being a conditional density to "the likelihood of the sample", a step also used in the maximum likelihood method.

• Second, $p(z)$, goes from being the marginal density to the "prior" density of $z$.

I am interested in this second mapping: can somebody lay out or suggest some literature where this step of going from a marginal density to a "prior" density is discussed in either technical or philosophical detail?

I’m guessing any examples from the literature would be rather "old,” because after decades of Bayesian statistics, calling $p(z)$ the "prior density" has become self-evident.

But to write "$p(z)$ is the prior density that incorporates all the prior beliefs/knowledge about $z$ except the sample at hand" raises the question: "where is the marginal density of a random variable defined in such a way? And if it is not defined in such a way (and it is not), then, what are the arguments that assert that the marginal density is indeed the proper object to represent what we want the "prior density" to represent?"

• See the references here. Also Savage (1954), The Foundations of Statistics. Commented Sep 6, 2018 at 10:30

The answer is that, to set up a problem that makes statistical sense, z and x are not the same kinds of quantities, despite the notational similarity.

• z is parameter that cannot be directly observed.
• x is observable data.

With those constraints in place, the likelihood piece is truly trivial, i.e. P(x|z), the probability of a set of observations given a parameter, is the definition of likelihood.

Statisticians have been arguing about the second part for centuries, but without using the "prior" terminology, P(z) is the probability that the parameter z has a certain value, as has been determined in the absence of the set of observations x. One condition where x definitely has not be incorporated into the determination of z is before the observations x are known, so the term "prior" is appropriate.

• Thanks for your input. The problem is that in the definition of the "marginal distribution", there is no special mention of some particular piece of information missing. Rather all "particular" information is missing. On the other hand, the "prior" focuses on the probability if the specific sample is missing from the information set. Essentially the prior is just another conditional probability distribution, not the marginal distribution... maybe Renyi's axiomatization that takes the conditional probability as the fundamental concept is what is needed here.. Commented Oct 16, 2018 at 17:50
• When you say "cannot be directly observed" how do you deal with different people "observing" different things? Also, the posterior $p(x|z)$ (ie infer $x$ given observed $z$) is just as valid mathematically as $p(z|x)$ is it not? Commented Jun 27, 2020 at 4:55

...what are the arguments that assert that the marginal density is indeed the proper object to represent what we want the "prior density" to represent?"

At an essential level, I think the argument is that this distribution is "prior" to the data because it does not condition on the data ---i.e., the marginal distribution of the parameters is inherently "prior" to the data because if it were not, it would need to condition on the data. This seems to me to be the essence of the prior/posterior difference.