Consistent estimator of the expectation of a conditional probability I'm stuck in a problem where I have distribution distribution $P(\boldsymbol{x})$, from which I know how to sample from (i.i.d.) and two functions of the random variable $\boldsymbol{x}$: $E(\boldsymbol{x})$ and $A(\boldsymbol{x})$ (both uninvertible). The aim is to compute the conditional mean of $A$ given $E$, $\mathbb{E}[A|E]$.
The question is how to do it. 
Specifically, I'm writing that
$$P(A|E) = 
\int P(\boldsymbol{x},A|E)d\boldsymbol{x} =
\int P(A|\boldsymbol{x},E)P(\boldsymbol{x}|E)d\boldsymbol{x} =
\frac{1}{P(E)}\int P(A|\boldsymbol{x},E)P(E|\boldsymbol{x})P(\boldsymbol{x}) d\boldsymbol{x}$$
I then use $P(A|\boldsymbol{x}) = \delta(A-A(\boldsymbol{x}))$ ($\delta$ is the Dirac-delta) to write
$$\mathbb{E}[A|E] = 
\int A P(A|E) dA = 
\frac{1}{P(E)}\int \int A \delta(A-A(\boldsymbol{x}) P(E|\boldsymbol{x})P(\boldsymbol{x}) d\boldsymbol{x} dA
$$
Integrating the $\delta(A-A(\boldsymbol{x}))$ out, and using $P(E|\boldsymbol{x}) = \delta(E-E(\boldsymbol{x}))$ leads to
$$\mathbb{E}[A|E] = \frac{1}{P(E)}\int A(\boldsymbol{x}) \delta(E - E(\boldsymbol{x})) P(\boldsymbol{x}) d\boldsymbol{x} \equiv \frac{I(E)}{P(E)}$$
Both $P(E)$ and $I(E)$ are consistently estimated (denoted as $\overline{P}$ and $\overline{I}$) by the calculation of a multidimensional integral using Monte Carlo. However:


*

*is $\overline{I}/\overline{P}$ a consistent estimator of $\mathbb{E}[A|E]$?

*How do I compute the error estimate of this estimator? $\overline{I}$ is not independent of $\overline{P}$...

*What is the error estimate of this estimator if $\boldsymbol{x}$ are now correlated?
 A: I do not know what functions you are using (I am assuming that they are not stochastic here) but this is the way I would deal with the problem:  


*

*Instead of defining $P(E|X)$ as a Dirac Delta, define it as a discrete probability distribution such that 
$$
P(E=E(x)|x)=\begin{cases}
1\;\;\mathrm{if}\;\;E=E(x) \\
0\;\;\mathrm{otherwise}
\end{cases}
$$  


This makes sense since because $E(x)$ is a function you know the value of $E$ with 100% certainty once you know $x$.  As suggested in your comment (A), you can express the above distribution as the Kronecker delta;
$$
\delta_{E,E(x)} = P(E=E(x)|x)
$$
 2. Using Bayes Theorem...
$$
P(X|E) =  \frac{P(E=E(x)|x)P(X)}{P(E)} = \delta_{E,E(x)} \cdot \frac{P(X)}{P(E)}
$$
The key here is that the support for $P(X|E)$ is a subset of the support for $P(X)$.  i.e. Let $x \in \mathcal{X}$ and $x|E \in \mathcal{X_E}$ where $\mathcal{X_E} \subseteq \mathcal{X}$.  Note that for this particular problem the Kronecker Delta $\delta_{x \in \mathcal{X_E}}$ is equivalent to  $\delta_{E,E(x)}$.*
You can then define 
$$
P(E) =\begin{cases}
 \sum_{x \in \mathcal{X_E}} P(x)\;\;\mathrm{if}\;\;\mathcal{X_E}\;\mathrm{is\;countable}\\
\;\\
\int_{\mathcal{X_E}} P(x)dx\;\;\mathrm{Otherwise}
\end{cases}
$$ 
With this notation you can also redefine your conditional expectation,
$$
\mathbb{E}[A|E] = \mathbb{E}[A|x \in \mathcal{X_E}]
$$
Once you know $E$ you should be able to define the set $\mathcal{X_E}$.  In the trivial case that $E(x)$ was invertible, $\mathcal{X_E}$ would only contain a single number.  Sampling from $P(X|E)$ can be accomplished with Monte-Carlo, Metropolis-Hastings, or direct calculation depending upon the nature of $\mathcal{X_E}$.
You can then write the conditional expectation as
$$
 \mathbb {E}[A|x \in \mathcal{X_E}] =\begin{cases}
\sum_{x \in \mathcal{X_E}} A(x) \frac{P(X)}{P(E)}\;\; \mathrm{if} \;\; \mathcal{X_E}\;\mathrm{is\;countable} \\ 
\; \\
\int_{\mathcal{X_E}} A(x) \frac{P(X)}{P(E)}dx \;\; \mathrm{otherwise}
\end{cases} 
$$
and under some regularity conditions this would be a consistent estimator.
To address your concerns in comment (B), it may very well be more computationally efficient to simulate a joint distribution $(A(x_i),E(x_i))$ if you want to estimate the conditional value of $A$ in respect to many $E^*$.  In such a setting you could either average $A(x_i)$ over a weighted sample (with larger weights assigned to values closer to $E^*$), or average $A$ over a smaller sub-sample interval around $E^*$. Whether or not this technique is efficient in the statistical sense depends a lot on the functions you are using. I think it would be a consistent estimator....but I am not entirely sure.
I guess part of the problem is that I do not know what functions you are using.  If, for example, $E(x) = x^2$ (totally trivial I know...) then it would be easy to implement the technique I discussed above.  With very complicated functions I could see how it would be a total pain, especially if you had to implement an MCMC for each individual  $E^*$.
