Efficiently drawing from non-parametrically estimated distribution Suppose I can estimate a distribution $G(x)$ as
$$ \hat G(x) = f(x, X)$$
where $X$ are my data points and $f$ is a known, but computationally heavy function.
Eventually I'm interested in
$$ h(\theta) = \int k(x, \theta) dG(x)$$
which I will have to evaluate many many times for different $\theta$. Thus, sampling from $G(x)$ repeatedly is very slow, since it involves repeated evaluation of $f(x,X)$. 
I'm thinking about pre-evaluating $\hat G(x)$ at a set of pre-determined points, e.g., some grid, and only sampling from there, but I'm worried that this might bias my results.
What can be done here?
 A: If you think about this as a numerical integration problem, you can refer to the extremely large literature on same.  As a starting point, if $k(\theta,x)dG(x)$ can be reasonably well approximated by some high degree polynomial, Gaussian quadrature may work very well with relatively few sample points required, e.g., 11 points will get you an exact integral for a 23-degree polynomial.  Since those points would remain the same for different values of $\theta$, you could indeed pre-compute $G$ at the appropriate values of $x$.  There are plenty of other quadrature and adaptive formulae that may be more-or-less efficient depending on the shape of $k(\theta, x)dG(x)$ over the range of $x$.
Note that different numerical integration formulae have different requirements for the range of $x$; these are usually satisfied by finding a transform of the original $x$ to a new variable that lies within the range and doesn't have unfortunate behavior within the range (e.g., virtually all the mass in the range $[-1,1]$ is concentrated between $[-0.005,-0.0049]$.)  Finding such an integration rule and associated transform is the art of numerical integration, which we can't get more into here since I have no idea what your function looks like.  However, off-the-shelf methods work well in most practical applications, so it may not be much of an issue for you.
A: Use a low-discrepancy sequence in your pre-computation.
https://en.wikipedia.org/wiki/Sobol_sequence
