Bayesian predictions from posterior parameter distributions

Question

I have two physical models $f(\theta)$ and $g(\theta)$ (not probability distributions) parameterized on the same set of parameters $\theta$. I also have data $y$ with measurement noise $\epsilon$ which can be used to obtain posterior estimates $p(\theta | y)$ through one of the models: \begin{align} y &= f(\theta) + \epsilon \\ \text{where}\ \epsilon &\sim \text{Normal}(0, \sigma) \end{align} so a Gaussian likelihood function can be defined: \begin{equation} p(y | \theta) \sim \exp\left(\frac{-(y - f(\theta))^2}{2\sigma^2} \right) \end{equation}

Questions:

1. Most of what I've read seems to be concerned with constructing posterior predictive distributions for new $y$ (denoting as $y_{\text{new}}$), as in $p(y_{\text{new}}|y) = \int p(y_{\text{new}}|\theta) p(\theta|y)d\theta$. But don't I just want the parameter-weighted distribution $f(\theta)p(\theta|y)$ without the measurement noise $\epsilon$ for the best estimate of $f(\theta)$?
2. Letting $z=g(\theta)$, is it necessary to specify a probability density function $p(z|\theta)$ so that new predictions of $z$ are defined as the marginal distribution $p(z|y) = \int p(z|\theta) p(\theta|y)d\theta$ (e.g., Robert, The Bayesian Choice, 2007)? Can I not just construct a probability distribution for the second physical model from $g(\theta)p(\theta|y)$? For instance using MCMC I could construct the probability distribution from the sequence $\{g(\theta_1), g(\theta_2), \ldots, g(\theta_t)\}$.

Answer 1

1. Most of what I've read seems to be concerned with constructing posterior predictive distributions for new $y$ (denoting as $y_{\text{new}}$), as in $p(y_{\text{new}}|y) = \int p(y_{\text{new}}|\theta) p(\theta|y)d\theta$. But don't I just want the parameter-weighted distribution $f(\theta)p(\theta|y)$ without the measurement noise $\epsilon$ for the best estimate of $f(\theta)$?

Expected value from the posterior distribution is "best" only in terms of minimizing squared error (for details, check The Bayesian Choice, already mentioned by you), but you could want instead look at other statistics, for example median of the posterior distribution (minimize absolute loss), or mode (maximum a posteriori estimate), etc. If you have estimate of the posterior distribution, you can easily estimate any of those quantities.

2. Letting $z=g(\theta)$, is it necessary to specify a probability density function $p(z|\theta)$ so that new predictions of $z$ are defined as the marginal distribution $p(z|y) = \int p(z|\theta) p(\theta|y)d\theta$ (e.g., Robert, The Bayesian Choice, 2007)? Can I not just construct a probability distribution for the second physical model from $g(\theta)p(\theta|y)$? For instance using MCMC I could construct the probability distribution from the sequence $\{g(\theta_1), g(\theta_2), \ldots, g(\theta_t)\}$.

By the law of unconscious statistician

$$ E[g(x)] = \int g(x)\,p(x)\, dx $$

If you have samples from the posterior distribution of $\tilde\theta_1,\tilde\theta_2,\dots,\tilde\theta_n \sim p(\theta|y)$, then to obtain the samples of their transformations, you just need to transform the samples; to get estimate of the expected value of the transformation of $\theta$, just calculate the empirical mean of the transformed MCMC samples.