Return to Answer

I'm going to start by cleaning up a little notational lack of precision in the first part of the question, then go on to the meat of it.

First, if $y \in \{0,1\}$, $y$ is, more precisely than stated in the question, distributed Bernoulli$(\theta)$, not Binomial$(n,\theta)$ for some arbitrary $n$. The two distributions are of course the same when $n=1$, but best to get rid of $n$ altogether if you don't use it subsequently.

On to the question! What the posterior distribution $p(\beta;\cdot)$ allows us to do is calculate an MCMC-based posterior for $\theta$ by sampling from the posterior distribution of $\beta$, then multiplying each sample $\beta$ by $X_{\text{new}}$ and calculating the inverse logit of the result (as in your second equation above.) This gives us a vector of samples of $\theta$ from the posterior distribution of $\theta$. Thanks to the Bernoulli assumption, these samples are samples from the posterior distribution of the expected value of $y_{\text{new}}$ as well as from the posterior distribution of the probability that $y_{\text{new}} = 1 | X_{\text{new}}, y$.

Note that we are not using the posterior predictive distribution to do this, just the posterior distribution itself. Your confusion stems from the fact that you tried to work with the posterior predictive distribution of $y_{\text{new}}$, which, for the problem you faced, was a step too far.

The samples of $\theta$ thus generated can be used in the usual MCMC ways in place of the full posterior distribution for $\theta$ or as inputs to a smoothing function of some sort (for calculating a smooth version of the posterior.) Either way, the sample-based posterior distribution contains the information you need to find (estimates of) the expected value of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$, (estimates of) quantiles of same, etc. For example, the posterior mean of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$ is just the mean of the sampled $\theta$ values, the 5th percentile is just the 5th percentile of the sampled $\theta$ values, etc.

I'm going to start by cleaning up a little notational lack of precision in the first part of the question, then go on to the meat of it.

First, if $y \in \{0,1\}$, $y$ is, more precisely than stated in the question, distributed Bernoulli$(\theta)$, not Binomial$(n,\theta)$ for some arbitrary $n$. The two distributions are of course the same when $n=1$, but best to get rid of $n$ altogether if you don't use it subsequently.

On to the question! What the posterior distribution $p(\beta|\cdot)$ allows us to do is calculate an MCMC-based posterior for $\theta$ by sampling from the posterior distribution of $\beta$, then multiplying each sample $\beta$ by $X_{\text{new}}$ and calculating the inverse logit of the result (as in your second equation above.) This gives us a vector of samples of $\theta$ from the posterior distribution of $\theta$. Thanks to the Bernoulli assumption, these samples are samples from the posterior distribution of the expected value of $y_{\text{new}}$ as well as from the posterior distribution of the probability that $y_{\text{new}} = 1 | X_{\text{new}}, y, X$.

Note that we are not using the posterior predictive distribution to do this, just the posterior distribution itself. Your confusion stems from the fact that you tried to work with the posterior predictive distribution of $y_{\text{new}}$, which, for the problem you faced, was a step too far.

The samples of $\theta$ thus generated can be used in the usual MCMC ways in place of the full posterior distribution for $\theta$ or as inputs to a smoothing function of some sort (for calculating a smooth version of the posterior.) Either way, the sample-based posterior distribution contains the information you need to find (estimates of) the expected value of $p(y_{\text{new}} = 1| X_{\text{new}}, y, X)$, (estimates of) quantiles of same, etc. For example, the posterior mean of $p(y_{\text{new}} = 1| X_{\text{new}}, y, X)$ is just the mean of the sampled $\theta$ values, the 5th percentile is just the 5th percentile of the sampled $\theta$ values, etc.

I'm going to start by cleaning up a little notational lack of precision in the first part of the question, then go on to the meat of it.

First, if $y \in \{0,1\}$, $y$ is, more precisely than stated in the question, distributed Bernoulli$(\theta)$, not Binomial$(n,\theta)$ for some arbitrary $n$. The two distributions are of course the same when $n=1$, but best to get rid of $n$ altogether if you don't use it subsequently.

On to the question! What the posterior distribution $p(\beta;\cdot)$ allows us to do is calculate an MCMC-based posterior for $\theta$ by sampling from the posterior distribution of $\beta$, then multiplying each sample $\beta$ by $X_{\text{new}}$. This gives us a vector of samples of $\theta$ from the posterior distribution of $\theta$. Thanks to the Bernoulli assumption, these samples are samples from the posterior distribution of the expected value of $y_{\text{new}}$ as well as from the posterior distribution of the probability that $y_{\text{new}} = 1 | X_{\text{new}}, y$.

Note that we are not using the posterior predictive distribution to do this, just the posterior distribution itself. Your confusion stems from the fact that you tried to work with the posterior predictive distribution of $y_{\text{new}}$, which, for the problem you faced, was a step too far.

The samples of $\theta$ thus generated can be used in the usual MCMC ways in place of the full posterior distribution for $\theta$ or as inputs to a smoothing function of some sort (for calculating a smooth version of the posterior.) Either way, the sample-based posterior distribution contains the information you need to find (estimates of) the expected value of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$, (estimates of) quantiles of same, etc. For example, the posterior mean of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$ is just the mean of the sampled $\theta$ values, the 5th percentile is just the 5th percentile of the sampled $\theta$ values, etc.

I'm going to start by cleaning up a little notational lack of precision in the first part of the question, then go on to the meat of it.

First, if $y \in \{0,1\}$, $y$ is, more precisely than stated in the question, distributed Bernoulli$(\theta)$, not Binomial$(n,\theta)$ for some arbitrary $n$. The two distributions are of course the same when $n=1$, but best to get rid of $n$ altogether if you don't use it subsequently.

On to the question! What the posterior distribution $p(\beta;\cdot)$ allows us to do is calculate an MCMC-based posterior for $\theta$ by sampling from the posterior distribution of $\beta$, then multiplying each sample $\beta$ by $X_{\text{new}}$ and calculating the inverse logit of the result (as in your second equation above.) This gives us a vector of samples of $\theta$ from the posterior distribution of $\theta$. Thanks to the Bernoulli assumption, these samples are samples from the posterior distribution of the expected value of $y_{\text{new}}$ as well as from the posterior distribution of the probability that $y_{\text{new}} = 1 | X_{\text{new}}, y$.

Note that we are not using the posterior predictive distribution to do this, just the posterior distribution itself. Your confusion stems from the fact that you tried to work with the posterior predictive distribution of $y_{\text{new}}$, which, for the problem you faced, was a step too far.

The samples of $\theta$ thus generated can be used in the usual MCMC ways in place of the full posterior distribution for $\theta$ or as inputs to a smoothing function of some sort (for calculating a smooth version of the posterior.) Either way, the sample-based posterior distribution contains the information you need to find (estimates of) the expected value of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$, (estimates of) quantiles of same, etc. For example, the posterior mean of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$ is just the mean of the sampled $\theta$ values, the 5th percentile is just the 5th percentile of the sampled $\theta$ values, etc.

I'm going to start by cleaning up a little notational confusion in the first part of the question, then go on to the meat of it.

First, if $y \in \{0,1\}$, $y$ is, more precisely than stated in the question, distributed Bernoulli$(\theta)$, not Binomial$(n,\theta)$ for some arbitrary $n$. The two distributions are of course the same when $n=1$, but best to get rid of $n$ altogether if you don't use it subsequently.

On to the question! What the posterior distribution $p(\beta;\cdot)$ allows us to do is calculate an MCMC-based posterior for $\theta$ by sampling from the posterior distribution of $\beta$, then multiplying each sample $\beta$ by $X_{\text{new}}$. This gives us a vector of samples of $\theta$ from the posterior distribution of $\theta$. Thanks to the Bernoulli assumption, these samples are samples from the posterior distribution of the expected value of $y_{\text{new}}$ as well as from the posterior distribution of the probability that $y_{\text{new}} = 1 | X_{\text{new}}, y$.

Note that we are not using the posterior predictive distribution to do this, just the posterior distribution itself. Your confusion stems from the fact that you tried to work with the posterior predictive distribution of $y_{\text{new}}$, which, for the problem you faced, was a step too far.

The samples of $\theta$ thus generated can be used in the usual MCMC ways in place of the full posterior distribution for $\theta$ or as inputs to a smoothing function of some sort (for calculating a smooth version of the posterior.) Either way, the sample-based posterior distribution contains the information you need to find (estimates of) the expected value of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$, (estimates of) quantiles of same, etc. For example, the posterior mean of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$ is just the mean of the sampled $\theta$ values, the 5th percentile is just the 5th percentile of the sampled $\theta$ values, etc.

I'm going to start by cleaning up a little notational lack of precision in the first part of the question, then go on to the meat of it.

First, if $y \in \{0,1\}$, $y$ is, more precisely than stated in the question, distributed Bernoulli$(\theta)$, not Binomial$(n,\theta)$ for some arbitrary $n$. The two distributions are of course the same when $n=1$, but best to get rid of $n$ altogether if you don't use it subsequently.

On to the question! What the posterior distribution $p(\beta;\cdot)$ allows us to do is calculate an MCMC-based posterior for $\theta$ by sampling from the posterior distribution of $\beta$, then multiplying each sample $\beta$ by $X_{\text{new}}$. This gives us a vector of samples of $\theta$ from the posterior distribution of $\theta$. Thanks to the Bernoulli assumption, these samples are samples from the posterior distribution of the expected value of $y_{\text{new}}$ as well as from the posterior distribution of the probability that $y_{\text{new}} = 1 | X_{\text{new}}, y$.

Note that we are not using the posterior predictive distribution to do this, just the posterior distribution itself. Your confusion stems from the fact that you tried to work with the posterior predictive distribution of $y_{\text{new}}$, which, for the problem you faced, was a step too far.

The samples of $\theta$ thus generated can be used in the usual MCMC ways in place of the full posterior distribution for $\theta$ or as inputs to a smoothing function of some sort (for calculating a smooth version of the posterior.) Either way, the sample-based posterior distribution contains the information you need to find (estimates of) the expected value of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$, (estimates of) quantiles of same, etc. For example, the posterior mean of $p(y_{\text{new}} = 1; X_{\text{new}}, y)$ is just the mean of the sampled $\theta$ values, the 5th percentile is just the 5th percentile of the sampled $\theta$ values, etc.