2
$\begingroup$

I have a $N$-dimensional (normal) random vector $\mathbf{X}$, where $N$ is large, and a function $f : \mathbb{R}^N \to \mathbb{R}$. My goal is to compute $\operatorname{E}[f(X)]$ or at least approximate it.

Things I've tried:

  • Monte Carlo: I sample $\mathbf{X}$ and compute the sample mean of $f(\mathbf{X})$
  • Numerical integration: the expected value is just an $N$-dimensional integral, so just evaluate it using numerical methods
  • Replace $f$ by its second (or higher?) order Taylor polynomial and use the fact that I know $\mathbf{X}$'s moments. (see here)

The first two methods seem prohibitively slow, appear to grow exponentially. The third is fast enough, but I have no idea how accurate I am.

What are the best/better ways to go about this problem?

Edit: some additional info about the $f$:

$$f(\mathbf{x}) = u(\langle \mathbf{v}, e^\mathbf{x} \rangle)$$

where $\mathbf{v}$ is a fixed vector, $u : \mathbb{R} \to \mathbb{R}$ is analytic and maybe even increasing and $e^\mathbf{x}$ is done pointwise.

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ What do you know about $f$? $\endgroup$
    – whuber ♦
    Commented Jun 22, 2015 at 14:12
  • $\begingroup$ One specific example I'm thinking about is $f(\mathbf{x}) = u(\langle \mathbf{v}, \mathbf{x} \rangle)$ where $\mathbf{v}$ is a fixed vector and $u : \mathbb{R} \to \mathbb{R}$ is analytic. The special case of increasing $u$ is also of interest. $\endgroup$
    – user80379
    Commented Jun 22, 2015 at 14:29
  • $\begingroup$ Thank you! If you would edit your post to include that information, you will get answers that are much more useful than otherwise. In particular, because $\langle\mathbf{v},\mathbf{x}\rangle$ has a univariate Normal distribution you are effectively in the case $N=1$, which completely changes the nature of the question. $\endgroup$
    – whuber ♦
    Commented Jun 22, 2015 at 14:32
  • $\begingroup$ I actually gave the wrong $f$, sorry. I meant to write $f(\mathbf{x}) = u (\langle \mathbf{v}, e^\mathbf{x} \rangle)$ where $e^\mathbb{x}$ is done pointwise $\endgroup$
    – user80379
    Commented Jun 22, 2015 at 14:35
  • 1
    $\begingroup$ My correlation matrix is non-trivial, unfortunately. $\endgroup$
    – user80379
    Commented Jun 22, 2015 at 14:41

1 Answer 1

Reset to default
1
$\begingroup$

Regarding Monte Carlo, a.k.a. stochastic, simulation, you can use a variance reduction method to potentially greatly decrease the number of simulation replications (trials) you need to perform to get an adequate accuracy in the expected value you're trying to estimate.

If your Taylor approximation, which you can compute without resorting to random numbers, is strongly (positively) correlated, when evaluated by stochastic simulation using the same random numbers (sample values of X) as for the function for which you are really trying to estimate Ef(X), then control variates (control variables) https://en.wikipedia.org/wiki/Control_variates (this is a terrible reference though, you really need to read a several page long section is a stochastic simulation book) can reduce the variance, and therefore the number of replications needed to get an accurate enough estimate, by a huge amount (can be orders of magnitude if correlation is high enough).

On each replication of the simulation, form the sample value of f(X) as well as sample value of the Taylor approximation with argument value X. Conduct a pilot run of a small number of replications to estimate the covariance between the two (or run a linear regression to do so). Then perform the full-scale run in which each sample of f(X) is corrected by a multiple, determined from the pilot run, of the difference between {the sample value of the Taylor approximation with argument value X{ and {the numerically determined (without random numbers) value of the expected value of the Taylor approximation}.

Due to the imperfect covariance estimate calculated from the pilot run results, the overall control variable method estimator of Ef(X) may be slightly biased, but variance reduction should more than make up for it, so as to decrease Mean Squared Error (MSE). The inaccuracy (bias) in the Taylor approximation does NOT contribute to any bias in the estimate of EF(X).

Edit: See section 5.5 of http://www.cs.fsu.edu/~mascagni/Hammersley-Handscomb.pdf for a brief but better introduction than Wikipedia.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.