In the context of Bayesian logistic regression, outcomes $y$ are binary (discrete) and covariates $X$ are given. We assume in particular:

$$ y | x \sim Bin(n,\theta) \newcommand{\new}{{\rm new}}$$

where we model $\theta$ as:

$$\theta = {\rm logit}^{-1}(X\beta) $$

with parameters $\beta$. Using a prior distribution $p(\alpha)$ for $\beta$ we come to the posterior:

$$p(\beta|y,X;\alpha) \propto p(y|X,\beta) p(\alpha)$$

where $p(y|X,\beta)$ the conditional likelihood.

We can now make two types of predictions for a new observation $X_{\new}$, one for $y_{\new} | X_{\new}, X, y \in \{0,1\} $ and one for $p(y_{\new} = 1| X_{\new},X, y) \in (0,1)$.

My question is: How can one make these predictions and quantify the uncertainty in the predictions given the data from the sample $(X,y)$?

I believe the posterior predictive distribution (ppd) is $$p(y_{new}|X,y,X_{new}) = \int p(y_{\new}|X_{\new},X,\beta) p(\beta|y,X) d\beta.$$

We can thus sample from the ppd $D$ times leading to $d=1,...,D$ samples $\tilde{y}_{\new,d}$ from $p(y_{\new}|X,\theta)$. Thus it is clear how to generate predictions for $y_{\new}$, but it is not clear to me how to generate a prediction for $p(y_{\new} = 1| X_{\new},y)$ and quantify its uncertainty.

I believe it is not central to my question, which prior to choose for $\beta$. However, a normal prior would be okay, so that samples from the posterior can be generated by MCMC or Laplace approximation.

In the context of Bayesian logistic regression, outcomes $y$ are binary (discrete) and covariates $X$ are given. We assume in particular:

$$ p(y | x, \theta) = Bin(n,\theta) \newcommand{\new}{{\rm new}}$$

where we model $\theta$ as:

$$\theta = {\rm logit}^{-1}(X\beta) $$

so

$$ p(y | x, \beta) = Bin(n,{\rm logit}^{-1}(X\beta)) \newcommand{\new}{{\rm new}}$$

with regression parameters $\beta$. 

Using a prior distribution $p(\alpha)$ for $\beta$ we come to the posterior:

$$p(\beta|y,X;\alpha) \propto p(y|X,\beta) p(\alpha)$$

where $p(y|X,\beta)$ the conditional likelihood.

We can now make two types of predictions for a new observation $X_{\new}$, one for $y_{\new} | X_{\new}, X, y \in \{0,1\} $ and one for $p(y_{\new} = 1| X_{\new},X, y) \in (0,1)$.

My question is: How can one make these predictions and quantify the uncertainty in the predictions given the data from the sample $(X,y)$?

I believe the posterior predictive distribution (ppd) is $$p(y_{new}|X,y,X_{new}) = \int p(y_{\new}|X_{\new},X,\beta) p(\beta|y,X) d\beta.$$

We can thus sample from the ppd $D$ times leading to $d=1,...,D$ samples $\tilde{y}_{\new,d}$ from $p(y_{\new}|X,\theta)$. Thus it is clear how to generate predictions for $y_{\new}$, but it is not clear to me how to generate a prediction for $p(y_{\new} = 1| X_{\new},y)$ and quantify its uncertainty.

I believe it is not central to my question, which prior to choose for $\beta$. However, a normal prior would be okay, so that samples from the posterior can be generated by MCMC or Laplace approximation.

In the context of Bayesian logistic regression, outcomes $y$ are binary (discrete) and covariates $X$ are given. We assume in particular:

$$ y | x \sim Bin(n,\theta) \newcommand{\new}{{\rm new}}$$

where we model $\theta$ as:

$$\theta = {\rm logit}^{-1}(X\beta) $$

with parameters $\beta$. Using a prior distribution $p(\alpha)$ for $\beta$ we come to the posterior:

$$p(\beta|y,X;\alpha) \propto p(y|X,\beta) p(\alpha)$$

where $p(y|X,\beta)$ the conditional likelihood.

We can now make two types of predictions for a new observation $X_{\new}$, one for $y_{\new} | X_{\new} \in \{0,1\} $ and one for $p(y_{\new} = 1| X_{\new},y) \in (0,1)$.

My question is: How can one make these predictions and quantify the uncertainty in the predictions given the data from the sample $(X,y)$?

I believe the posterior predictive distribution (ppd) is $$p(y_{new}|y,X_{new}) = \int p(y_{\new}|X_{\new},\beta) p(\beta|y,X) d\beta.$$

We can thus sample from the ppd $D$ times leading to $d=1,...,D$ samples $\tilde{y}_{\new,d}$ from $p(y_{\new}|X,\theta)$. Thus it is clear how to generate predictions for $y_{\new}$, but it is not clear to me how to generate a prediction for $p(y_{\new} = 1| X_{\new},y) $ and it is also not clear how to quantify the uncertainty. Ideally I would want credible intervals around $p(y_{\new} = 1| X_{\new},y)$ and a measure of misclassification probability for $\tilde{y}_{\new}$. The uncertainty in these statistics must be created by sampling variability as $X$ and $y$ are of size $n$, which may be small. So any specific estimate for $p(y_{\new} = 1| X_{\new},y)$ must be imprecise and the sample $\tilde{y}_{\new}$ is subject to error.

In the context of Bayesian logistic regression, outcomes $y$ are binary (discrete) and covariates $X$ are given. We assume in particular:

$$ y | x \sim Bin(n,\theta) \newcommand{\new}{{\rm new}}$$

where we model $\theta$ as:

$$\theta = {\rm logit}^{-1}(X\beta) $$

with parameters $\beta$. Using a prior distribution $p(\alpha)$ for $\beta$ we come to the posterior:

$$p(\beta|y,X;\alpha) \propto p(y|X,\beta) p(\alpha)$$

where $p(y|X,\beta)$ the conditional likelihood.

We can now make two types of predictions for a new observation $X_{\new}$, one for $y_{\new} | X_{\new}, X, y \in \{0,1\} $ and one for $p(y_{\new} = 1| X_{\new},X, y) \in (0,1)$.

My question is: How can one make these predictions and quantify the uncertainty in the predictions given the data from the sample $(X,y)$?

I believe the posterior predictive distribution (ppd) is $$p(y_{new}|X,y,X_{new}) = \int p(y_{\new}|X_{\new},X,\beta) p(\beta|y,X) d\beta.$$

We can thus sample from the ppd $D$ times leading to $d=1,...,D$ samples $\tilde{y}_{\new,d}$ from $p(y_{\new}|X,\theta)$. Thus it is clear how to generate predictions for $y_{\new}$, but it is not clear to me how to generate a prediction for $p(y_{\new} = 1| X_{\new},y)$ and quantify its uncertainty. 

I believe it is not central to my question, which prior to choose for $\beta$. However, a normal prior would be okay, so that samples from the posterior can be generated by MCMC or Laplace approximation.

In the context of Bayesian logistic regression, outcomes $y$ are binary (discrete) and covariates $X$ are given. We assume in particular:

$$ y | x \sim Bin(n,\theta)$$

where we model $\theta$ as:

$$\theta = logit^{-1}(X\beta) $$

with parameters $\beta$. Using a prior distribution $p(\alpha)$ for $\beta$ we come to the posterior:

$$p(\beta|y,X;\alpha) \propto p(y|X,\beta) p(\alpha)$$

where $p(y|X,\beta)$ the conditional likelihood.

We can now make two types of predictions for a new observation $X_{new}$, one for $y_{new} | X_{new} \in \{0,1\} $ and one for $p(y_{new} = 1| X_{new},y) \in (0,1)$.

My question is how to make these predictions and quantify the uncertainty in the predictions given the data from the sample $(X,y)$?

I believe the posterior predictive distribution (ppd) is $$p(y_{new}|y,X_{new}) = \int p(y_{new}|X_{new},\beta) p(\beta|y,X) d\beta.$$

We can thus sample from the ppd $D$ times leading to $d=1,...,D$ samples $\tilde{y}_{new,d}$ from $p(y_{new}|X,\theta)$. Thus it is clear how to generate predictions for $y_{new}$, but it is not clear to me how to generate a prediction for $p(y_{new} = 1| X_{new},y) $ and it is also not clear how to quantify the uncertainty. Ideally I would want credible intervals arround $p(y_{new} = 1| X_{new},y)$ and a measure of misclassification probability for $\tilde{y}_{new}$. The uncertainty in these statistics must be created by sampling variability as $X$ and $y$ are of size $n$, which may be small. So any specific estimate for $p(y_{new} = 1| X_{new},y)$ must be imprecise and the sample $\tilde{y}_{new}$ is subject to error.

In the context of Bayesian logistic regression, outcomes $y$ are binary (discrete) and covariates $X$ are given. We assume in particular:

$$ y | x \sim Bin(n,\theta) \newcommand{\new}{{\rm new}}$$

where we model $\theta$ as:

$$\theta = {\rm logit}^{-1}(X\beta) $$

with parameters $\beta$. Using a prior distribution $p(\alpha)$ for $\beta$ we come to the posterior:

$$p(\beta|y,X;\alpha) \propto p(y|X,\beta) p(\alpha)$$

where $p(y|X,\beta)$ the conditional likelihood.

We can now make two types of predictions for a new observation $X_{\new}$, one for $y_{\new} | X_{\new} \in \{0,1\} $ and one for $p(y_{\new} = 1| X_{\new},y) \in (0,1)$.

My question is: How can one make these predictions and quantify the uncertainty in the predictions given the data from the sample $(X,y)$?

I believe the posterior predictive distribution (ppd) is $$p(y_{new}|y,X_{new}) = \int p(y_{\new}|X_{\new},\beta) p(\beta|y,X) d\beta.$$

We can thus sample from the ppd $D$ times leading to $d=1,...,D$ samples $\tilde{y}_{\new,d}$ from $p(y_{\new}|X,\theta)$. Thus it is clear how to generate predictions for $y_{\new}$, but it is not clear to me how to generate a prediction for $p(y_{\new} = 1| X_{\new},y) $ and it is also not clear how to quantify the uncertainty. Ideally I would want credible intervals around $p(y_{\new} = 1| X_{\new},y)$ and a measure of misclassification probability for $\tilde{y}_{\new}$. The uncertainty in these statistics must be created by sampling variability as $X$ and $y$ are of size $n$, which may be small. So any specific estimate for $p(y_{\new} = 1| X_{\new},y)$ must be imprecise and the sample $\tilde{y}_{\new}$ is subject to error.