5
$\begingroup$

In Doing Bayesian Data Analysis 2ed, by Kruschke, in chapter 10, we get two equations (10.1, 10.2) for which no hint as to how they are obtained is given...

How does one get the second equality in the following decomposition$$p(\theta_1,\theta_2,...,m|D)= \frac{p(D|\theta_1,\theta_2,...,m)p(\theta_1,\theta_2,...,m)}{\sum_m \int p(D|\theta_1,\theta_2,...,m)p(\theta_1,\theta_2,...,m)\ d\theta_m}=\frac{\prod_m p_m(D|\theta_m,m)p_m(\theta_m|m)p(m)}{\sum_m \int \prod_m p_m(D|\theta_m,m)p_m(\theta_m|m)p(m)\ d\theta_m}$$ ?

I would think that $p(D|\theta_1,\theta_2,...,m)= p_m(D|\theta_m,m)$? I would say there's a typo in the first equality, and that the second is wrong... But I'm the one most probably wrong.

Improve this question
$\endgroup$
5
  • $\begingroup$ The first one is just Bayes' rule for the variables $D$ and $(\theta_1,...,m)$. The second looks like some decompostion is assumed, maybe the author assumes something? Some background would be helpful, since I don't have access to that book. $\endgroup$
    – Yair Daon
    Commented Jul 20, 2017 at 18:32
  • 1
    $\begingroup$ @YairDaon On the first equality, why is the integral on theta_m and not on all the thetas? $\endgroup$
    – An old man in the sea.
    Commented Jul 20, 2017 at 19:32
  • 1
    $\begingroup$ Great point, my bad. I think that since this is a model selection problem, given $m$ the likelihood does not depend on other $\theta$s. If you give more context we can be sure. $\endgroup$
    – Yair Daon
    Commented Jul 21, 2017 at 3:47
  • 1
    $\begingroup$ Regarding the second equality, something seems off. After the first equality, m is a bona fide variable - it has some value. But after the second equality, it is only a dummy variable - it takes all values (equivalently, takes none). I bet there is some sort of typo, maybe the author has an errata page in their website. $\endgroup$
    – Yair Daon
    Commented Jul 21, 2017 at 4:06
  • $\begingroup$ @YairDaon I've given all the context that is given in the book, and the errata doesn't have this page... ;) $\endgroup$
    – An old man in the sea.
    Commented Jul 21, 2017 at 8:11

1 Answer 1

Reset to default
3
$\begingroup$

I agree that the second formula has an issue, first because the $m$ on the left of the (first) equality sign is a fixed integer, while the $m$ on the right of the (second) equality sign is a running index. (Even more incoherent is the denominator where there are two $m$ indices!) And second because indeed$$p(D|\theta_1,\theta_2,m)=p_m(D|\theta_m)$$Hence it should be that $$p(\theta_1,\theta_2,\ldots,m|D)=\dfrac{p_m(D|\theta_m)p(m)\prod_{i\ne m}p(\theta_i|m)}{\sum_\mu p_\mu(D|\theta_\mu)p(\mu)\prod_{i\ne \mu}p(\theta_i|\mu)}$$This is a most interesting formula because it shows that a prior on $\theta_i$ must defined in all models for the joint distribution $p(\theta_1,\theta_2,\ldots,m|D)$ to make sense. In my opinion, this is not coherent with model choice: a parameter $\theta_i$ only exists when the model index is equal to $i$. To introduce the value of a model parameter within another model when this parameter has no connection with the data $D$ is not coherent. Note however that such distributions (called pseudo-priors) have been used for computational purposes, the most well-known reference being Carlin and Chib (1995) who devised an alternative to reversible jump MCMC by completing the posterior distribution into a fully joint $p(\theta_1,\theta_2,\ldots,m|D)$. But this was solely for computational reasons, there was no inferential meaning to those distributions [which could even depend on the data].

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ Yes, Xi'an is right w.r.t. details. But let me try to ameliorate a little. First, the two uses of $m$ are explicitly pointed out immediately after Eqn 10.2 in the book. Second, in principle the full joint distribution could be defined, even though in practice it's not except with pseudo-priors which are discussed 13 pages later in the book. In any case, thank you Christian for clarifying here! $\endgroup$
    – John K. Kruschke
    Commented Jul 26, 2017 at 19:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.