0
$\begingroup$

This is the first time I've come across the term posterior "ordinate" and I'm not sure what it means. I understand what the posterior density of some model parameters given some data refers to, but this seems to be different. I came across it reading this paper: http://www.tandfonline.com/doi/abs/10.1198/016214501750332848 has anyone ever heard of it before.

Given the context it is used in, could it just refer to the un-normalised posterior density?

UPDATE: The context of this is for calculating the marginal likelihood of bayesian statistical models, and their Bayes factors. It is related to this question Bayesian Information Criterion and Basic Marginal likelihood identity

To calculate the Bayes factor between two models $M_1$ and $M_2$ we need to calculate their marginal likelihoods $m(\boldsymbol{y}|M_i)$

$$ m(\boldsymbol{y}|M_0) = \int f(\boldsymbol{y}|M_0,\boldsymbol{\theta}_0)\pi(\boldsymbol{\theta}_0|M_0) dx $$

which can be written as

$$ m(\boldsymbol{y}|M_0) = \frac{f(\boldsymbol{y}|M_0,\boldsymbol{\theta}_0)\pi(\boldsymbol{\theta}_0|M_0)}{\pi(\boldsymbol{\theta}_0|\boldsymbol{y},M_0)}$$

which the writer calls the basic marginal likelihood identity though I haven't heard of it elsewhere. To explain the variables, $\boldsymbol{y}$ seem to be the "data", while $\boldsymbol{\theta}_0$ are the model parameters. I would call $f(\boldsymbol{y}|M_0,\boldsymbol{\theta}_0)$ the likelihood (data|parameters), and $\pi(\boldsymbol{\theta}_0|M_0)$ seem to be the priors on the model parameters. The thing on the bottom looks very much like the joint posterior distribution of the model parameters, however he calls that the posterior ordinate instead.

Since I've gone into detail, I would like to expand my question.

The paper is essentially deriving a way to calculate marginalised likelihoods from the normal output that a Metropolis-Hastings sampler would produce, i.e a representative sample of the joint posterior $\pi(\boldsymbol{\theta}_0|\boldsymbol{y},M_0)$. However, to do draw this sample, and run the MetroHastings sampler, we must have a way to calculate $\pi(\boldsymbol{\theta}_0|\boldsymbol{y},M_0)$ at some point $\boldsymbol{\theta}_0$. Though it's possible we don't have the proper normalising constant, which is where my previous assumption came from.

But essentially, if we can directly calculate the joint posterior, then surely using this basic marginal likelihood identity to calculate the marginal likelihood is as trivial as putting the numbers in. However, this doesn't seem to be the case and they use a much more complicated way which I am trying to look into.

$\endgroup$
  • 1
    $\begingroup$ Please provide more details and context. Exactly how is the term used? Give a quote and excerpt. (Moreover, the article you linked is behind a paywall.) $\endgroup$ – John K. Kruschke Jan 13 '17 at 14:42
  • $\begingroup$ Ah sorry about that, it doesn't seem to be on arxiv. I'll update my main post with more info, I was just wondering if it is a common statistical term. $\endgroup$ – Limok Palantaemon Jan 13 '17 at 14:56
  • $\begingroup$ I hope the update helps. $\endgroup$ – Limok Palantaemon Jan 13 '17 at 15:25
  • $\begingroup$ How are the two displays identical? One is marginalized over $x$ (whatever that is), and the other is not. $\endgroup$ – AdamO Jan 13 '17 at 16:40
  • $\begingroup$ If you rearrange the bottom display it looks like Bayes' law. I'll try and find his derivation of it, must be in another paper. Though I'm not used to the notation. $\endgroup$ – Limok Palantaemon Jan 15 '17 at 12:03
0
$\begingroup$

Just answering in case someone else was also wondering about this. Upon further reading, it seems that the answer is that the posterior ordinate and posterior density refer to the same thing, it seems to be some old convention.

As to the second part of my update, I was wondering why given

$$\pi(\boldsymbol{\theta}_0|\boldsymbol{y},M_0) \propto f(\boldsymbol{y}|M_0,\boldsymbol{\theta}_0)\pi(\boldsymbol{\theta}_0|M_0)$$

we can't just immediately subsitute it into the marginalised likelihood equation (we of course need to know the above, since we need it to run the M-H sampler).

The answer is that in general it is difficult to calculate the normalisation constant of the above expression. And while you don't need the normalisation constant for Metropolis-Hastings (so use the RHS), you do need it for calculating the marginalized likelihood (for which you need the LHS). Also, the normalisation constant is the marginalized likelihood itself, so if you know it the rest would be trivial.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.