1
$\begingroup$

I am having some difficulties when trying to do a little bit of algebra from Example 7.11 from the book "Introducing Monte Carlo Methods with R: Robert & Casella"

The example relates to logistic regression using a slice sampler. The model is (omitting $i$ index)

$$y \sim Bern(p(x)), \qquad p(x) = \frac{\exp(a + bx)}{1 + \exp(a + bx)}$$ And assuming flat priors on (a,b) the posterior distribution is given by

$$ p(a,b) = \prod_{i=1}^n \frac{\exp(y_i(a + bx_i))}{1 + \exp(a + bx_i)}$$

The first step of the slice sampler is given as

$$ U_i \sim\mathcal U \Bigg(0, \frac{\exp(y_i(a + bx_i))}{1 + \exp(a + bx_i)}\Bigg) $$ and the second as

$$ \mathcal U\left\{ (a,b): y_i(a+bx_i) > \log \frac{u_i}{1-u_i}\quad i=1,\ldots,n \right\} $$ I am struggling with this second step. I cannot get these values if I work backwards:

\begin{align} y(a+bx) &> \log u/(1-u) \\ \exp(y(a+bx)) &> u/(1-u) \\ (1-u)*\exp(y(a+bx)) &> u \\ \exp(y(a+bx)) &> u + u\exp(y(a+bx)) \\ \frac{\exp(y(a+bx))}{1 + \exp(y(a+bx))} &> u \\ \end{align}

The final term on the left-hand side does not equal the posterior distribution,whic I belive should be of the form $\{x: f(x) \geq U_i\}$. Where is my mistake please?

$\endgroup$
1
$\begingroup$

Sorry that this section proves a challenge! (I corrected the question to indicate that the Uniform on $(a,b)$ is imposing the inequality for all $i$'s, as this is (too) implicit in the book.) There is alas a typo in the book when defining the set (second displayed formula on p.220), not a mistake in your reasoning!

The constraint on $(a,b)$ is thus that for all $1\le i\le n$, $$\frac{\exp(y_i(a + bx_i))}{1 + \exp(a + bx_i)}\ge u_i$$ hence when $y_i=0$ $$u_i^{-1}\ge 1 + \exp(a + bx_i)$$ or $$\log\frac{1-u_i}{u_i}\ge a + bx_i$$ and when $y_i=1$ $$1-u_i\ge \frac1{1 + \exp(a + bx_i)}$$ or $$\log\frac{u_i}{1-u_i}\le a + bx_i$$ Meaning that $$ \max_{y_i=1}\left\{ \log\frac{u_i}{1-u_i}-bx_i\right\} \le a \le \min_{y_i=0} \left\{ \log\frac{1-u_i}{u_i}-bx_i\right\}$$ and the same for $b$. Thus the slice distributions (and the R code below) remain valid.

Apologies!!! I went back to our (LaTeX) source file for this chapter on Gibbs samplers and found that we had written

...variables. Generating a uniform distribution over the set $$ \bigcap_{i=1} \left\{ (a,b)\,:\ (-1)^{y_i}(a+bx_i) > \log\frac{u_i}{1-u_i} \right\} $$ being rather unwieldy, we can further decompose the uniform simulation by consecutively simulating...

Which is correct. For some reason I can neither fathom nor remember, both the intersection and power terms did not make it to the printed versions... (Including the French and Japanese translations!)

$\endgroup$
2
  • 1
    $\begingroup$ Dear Xi'an, thank you for working through this for me -- I find these extra details really useful.. $\endgroup$ – user2957945 Jan 15 at 12:57
  • 1
    $\begingroup$ @user2957945: sorry about this and the waste of time it induced!!! $\endgroup$ – Xi'an Jan 15 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.