4
$\begingroup$

Ok, this feels like it should be pretty basic, but maybe I'm missing the right terminology to google this.

Suppose I have a (non-analytical) function $f(x)$ that I have evaluated for a large number of uniform values $x_i \in [-1, 1]$. That is, I have uniform samples of points $(x_i, f_i=f(x_i))$.

Now I want to know what the expectation value $\bar{f}$ of $f$ would be if the values of $x$ were drawn from a Normal distribution of width $\sigma \ll 1$. Ultimately, I want to do this for different values of $\sigma$, so that I can generate a plot of $\bar{f}(\sigma)$.

My first intuition was to simply multiply the values $f_i$ with a weight $w_i$ where $w_i$ is the value of the probability density function (PDF) for the normal distribution with $\sigma$. I would then normalize with $\sum w_i$ and average the weighted and normalized $f_i$. However, that does not seem to produce correct results.

I've also tried rejection sampling (taking a simple average of the $f_i$, but randomly excluding $f_i$ if a random value $r_i \in [0, 1] $ is smaller than the normalized PDF at $x_i$). This gives me good results, but it's quite slow.

Is there a better method than rejection sampling for this? It still seems to me like I should be able to average my original $f_i$ with some appropriate weighting, but maybe I'm wrong.

There's lots of methods for generating normal-distributed $x_i$ from uniform $x_i$, of course, but that's not what I want to do: I don't want to evaluate $f(x)$ for any new values $x_i$, so all I have are the existing $f_i$. My $x_i$ in this case are in fact random, but in principle it should also work for $x_i$ on a dense regular grid.

$\endgroup$
2
  • $\begingroup$ So you want $E\{f(X)\}$, where $X$ is normally distributed? It is just a simple integral, $\int f(x)g(x) dx$, where $g(x)$ is your normal density. What am I missing? $\endgroup$ Aug 3, 2021 at 0:15
  • $\begingroup$ In the continuous limit, that equation looks right to me, but what do I do with that? $\endgroup$ Aug 3, 2021 at 2:30

1 Answer 1

1
$\begingroup$

On principle you simply cannot estimate$$\int_{-\infty}^{\infty} f(x) \varphi_\sigma(x)\,\text dx$$based on $f(x_i)$'s with $x_i\in(-1,1)$ since there is not information for $f(x)$ when $x\not\in(-1,1)$...

Now, if $\sigma$ is small enough for$$\int_{-1}^{1} \varphi_\sigma(x)\,\text dx\approx 1$$you can consider$$\frac{2}{n}\sum_{i=1}^nf(x_i)\varphi_\sigma(x_i)$$as a pseudo-importance sampler but, depending on the function $f$, the approximation can completely go south, i.e. be completely unreliable: take an extreme example when $f$ is very large on $(-2,-1)$ and very small or null over $(-1,1)$: $$f(x)=\begin{cases} 1/\varphi_\sigma(x) &\text{when }x\in(-2,-1)\\ 0 &\text{elsewhere} \end{cases} $$ Then $$\int_{-\infty}^{\infty} f(x) \varphi_\sigma(x)\,\text dx= 1$$ while $$\frac{1}{n}\sum_{i=1}^nf(x_i)\varphi_\sigma(x_i)=0$$

Obviously, if the support of $f$ is $(-1,1)$ then $$\int_{-\infty}^{\infty} f(x) \varphi_\sigma(x)\,\text dx= \int_{-1}^{1} 2f(x) \varphi_\sigma(x)\,\frac{\text dx}{2}=2\mathbb E[f(U)\varphi_\sigma(U)]$$when $U\sim\mathcal U(-1,1)$.

$\endgroup$
6
  • 1
    $\begingroup$ Yes, I understand there could be a pathological $f(x)$. In this particular case, $f(x)$ can be considered 0 outside $[-1, 1]$ and also $\sigma$ is going to be sufficiently small so that $\int_{-1}^{1} \phi(x) dx \approx 0$. Just to make sure: $\phi_\sigma(x)$ the PDF for normal distribution, right? Then your answer would correspond to my initial intuition, except for the normalization with $\sum\phi_\sigma(x_i)$. Let me try that... $\endgroup$ Aug 3, 2021 at 10:25
  • $\begingroup$ If $f(x)\equiv 1$, the average $\bar{f}$ should be 1 as well (independent of $\sigma$), which doesn't seem to be what I'm getting with $\frac{1}{n} \sum f_i \phi(x_i)$ (or with $\frac{2}{n} \sum f_i \phi(x_i)$ ) $\endgroup$ Aug 3, 2021 at 15:44
  • $\begingroup$ Actually, I think it turns out that my original intuition was correct after all: $\left(\sum_{i=1}^{n} f_i \phi_\sigma(x_i)\right) / \sum_{i=1}^{n} \phi_\sigma(x_i)$ does seem to work! What was tripping me up was a complication that I left out of the description here: I actually have another "category" parameter in my data, and I want to do do the averaging for each category indpendently. If I do the averaging before a groupby for the category, I also have to divide by the number of categories. Or, do the groupby first. $\endgroup$ Aug 3, 2021 at 16:36
  • $\begingroup$ @xian If you agree with / double check the result and edit your response accordingly, I'll accept it as the answer $\endgroup$ Aug 3, 2021 at 16:38
  • $\begingroup$ No, my answer is correct, this is a standard importance sampling approximation. If you disagree please write another answer. $\endgroup$
    – Xi'an
    Aug 3, 2021 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.