I have a specific question about an important point made in [http://arxiv.org/pdf/1507.06173.pdf, Sec. 4]. To summarize, let's consider a signal model
$\overrightarrow{R} \sim p(\overrightarrow{R} \vert t)$,
where $\overrightarrow{R} \in \mathbb{R}^n$ collects some noisy sensor measurements and is modeled as a multivariate Gaussian random variable with mean
$\mathbb{E}[\overrightarrow{R} \vert t] = \overrightarrow{\mu}(t)$
and variance
$\mathbb{V}[\overrightarrow{R} \vert t] = \Sigma(\overrightarrow{\mu}(t))$.
The elements of $\overrightarrow{\mu}(t)$ and $\Sigma(\overrightarrow{\mu})$ are (known) nonlinear functions of $t$. The goal is to find an estimator for the unknown $t$. The authors propose to use the MCMC algorithm to estimate the posterior $p(t \vert \overrightarrow{R})$ given some prior $p(t)$ and then compute the Bayesian mean
$\displaystyle\hat{t}=\int t\, p(t \vert \overrightarrow{R})\, dt$
Since running MCMC would be too slow in a real time image processing application, they build a training set and then use a random forest classifier to make predictions as follows. First, a value $t_i$ is sampled from the prior, then $\overrightarrow{R}_i$ is sampled from the conditional distribution (likelihood):
$t_i \sim p(t)$,
$\overrightarrow{R}_i \sim p(\overrightarrow{R} \vert t_i)$.
Now the Bayesian mean is computed from the posterior:
$\displaystyle\hat{t}_i=\int t\, p(t \vert \overrightarrow{R}_i)\, dt$
This process is repeated to build the training set
$(\overrightarrow{R}_i, \displaystyle\hat{t}_i)\qquad i=0,\ldots,N$.
Now to my question: one could as well use the training set
$(\overrightarrow{R}_i, t_i)$
where $t_i$ is the value sampled from the prior, therefore avoiding to run MCMC. According to the authors this would increase the variance of the output of the random forest regression algorithm. Is there a formal way to prove that? In other words, how can I estimate the variance of the output produced by the regression algorithms obtained using the two different training sets?
