4
$\begingroup$

I am looking at pg. 257 of the textbook "Statistical computing with R" by Maria L. Rizzo, where an example is given of a simple model of stock returns:

We have 5 stocks, and at the end of a year of 250 trading days, we have a vector [$x_1$, $x_2$, ..., $x_5$], where $x_i$ is the number of days that stock $i$ was the best performing stock. This is a multinomial joint distribution, with probability vector $p = [\frac{1}{3}, \frac{1 - \beta}{3}, \frac{1 - 2\beta}{3}, \frac{2\beta}{3}, \frac{\beta}{3} ]$, with $\beta \in (0, 0.5)$.

Now, what I don't understand, is that in the textbook they now say that the posterior probability distribution of $\beta$, which now represents the parameter of the multinomial distribution here, is

$Pr[\beta | (x_1, ..., x_5)] = \frac{250!}{x_1!x_2!x_3!x_4!x_5!}p_1^{x_1}p_2^{x_2}p_3^{x_3}p_4^{x_4}p_5^{x_5}$

which is also the pmf (probability mass function) of the multinomial distribution, (given the parameter) i.e.

$Pr[(x_1, ..., x_5) | \beta] = \frac{250!}{x_1!x_2!x_3!x_4!x_5!}p_1^{x_1}p_2^{x_2}p_3^{x_3}p_4^{x_4}p_5^{x_5}$

According to Bayes

$Pr[\beta | (x_1, ..., x_5)] = \frac{Pr[(x_1, ..., x_5) | \beta]Pr[\beta]}{Pr[(x_1, ..., x_5)]}$

So the only way $Pr[\beta | (x_1, ..., x_5)] = Pr[ (x_1, ..., x_5) | \beta]$ then is if $Pr[\beta] = Pr[(x_1, ..., x_5)]$, but I am assuming here that $\beta$ is uniformly distributed, making $Pr[\beta]$ a constant. $Pr[(x_1, ..., x_5)]$ is not a constant, if I am correct, because of combinatorics.

So what am I missing here?

$\endgroup$
  • $\begingroup$ The data are fixed and so the (marginal) probability of the data is a constant. $\endgroup$ – mef Mar 6 '16 at 15:13
  • $\begingroup$ @mef: as a function of $(x_1,x_2,\ldots,x_5)$ the normalising constant of the posterior on $\beta$ varies for different samples $(x_1,x_2,\ldots,x_5)$. $\endgroup$ – Xi'an Mar 6 '16 at 15:24
2
$\begingroup$

While I do not have access to this book, I think the first equation is missing a proportionality constant, i.e., that the posterior on $\beta$ should be $$\begin{align*}\pi(\beta | (x_1, ..., x_5)) &\propto \dfrac{250!}{x_1!x_2!x_3!x_4!x_5!}p_1^{x_1}p_2^{x_2}p_3^{x_3}p_4^{x_4}p_5^{x_5}\\ &\propto p_2(\beta)^{x_2}p_3(\beta)^{x_3}p_4(\beta)^{x_4}p_5(\beta)^{x_5}\\&\propto \{1 - \beta\}^{x_2}\, \{1 - 2\beta\}^{x_3}\, \beta^{x_4}\, \beta^{x_5}\\&\propto \{1 - \beta\}^{x_2}\, \{1 - 2\beta\}^{x_3}\, \beta^{x_4+x_5}\end{align*}$$ with a manageable normalising constant since this is a polynomial in $\beta$. You are right that the marginal $p(x_1, ..., x_5)$ is not a constant. For instance, $$\begin{align}\int_0^{1/2} \{1 - \beta\}^{3}\, &\{1 - 2\beta\}^{6}\, \beta^{9} \text{d}\beta = (1/2)^{10}/10 - (15(1/2)^{11})/11 \\ &+ (33(1/2)^{12})/4 - 29(1/2)^{13} + (456(1/2)^{14})/7 - (484(1/2)^{15})/5\\ &+ 95(1/2)^{16} - (1008(1/2)^{17})/17 + (64(1/2)^{18})/3 - (64(1/2)^{19})/19 \\ &= 4.38\, 10^{-9} \end{align}$$ and $$\begin{align}\int_0^{1/2} \{1 - \beta\}^{1}\, \{1 - 2\beta\}^{2}\, \beta^{3} \text{d}\beta &= (1/2)^4/4 - (1/2)^5 + (4(1/2)^6)/3 - (4(1/2)^7)/7 \\ &= 7.44\, 10^{-4} \end{align}$$ For a similar problem with a Gibbs sampling resolution, see Tanner & Wong (1987) [also processed in our book, pp.347-348].

$\endgroup$
  • $\begingroup$ Excellent answer. Thanks for clearing things up (and giving some extra insight)! $\endgroup$ – Konrad Kapp Mar 6 '16 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.