# Calculate a normal average from uniform samples

Ok, this feels like it should be pretty basic, but maybe I'm missing the right terminology to google this.

Suppose I have a (non-analytical) function $$f(x)$$ that I have evaluated for a large number of uniform values $$x_i \in [-1, 1]$$. That is, I have uniform samples of points $$(x_i, f_i=f(x_i))$$.

Now I want to know what the expectation value $$\bar{f}$$ of $$f$$ would be if the values of $$x$$ were drawn from a Normal distribution of width $$\sigma \ll 1$$. Ultimately, I want to do this for different values of $$\sigma$$, so that I can generate a plot of $$\bar{f}(\sigma)$$.

My first intuition was to simply multiply the values $$f_i$$ with a weight $$w_i$$ where $$w_i$$ is the value of the probability density function (PDF) for the normal distribution with $$\sigma$$. I would then normalize with $$\sum w_i$$ and average the weighted and normalized $$f_i$$. However, that does not seem to produce correct results.

I've also tried rejection sampling (taking a simple average of the $$f_i$$, but randomly excluding $$f_i$$ if a random value $$r_i \in [0, 1]$$ is smaller than the normalized PDF at $$x_i$$). This gives me good results, but it's quite slow.

Is there a better method than rejection sampling for this? It still seems to me like I should be able to average my original $$f_i$$ with some appropriate weighting, but maybe I'm wrong.

There's lots of methods for generating normal-distributed $$x_i$$ from uniform $$x_i$$, of course, but that's not what I want to do: I don't want to evaluate $$f(x)$$ for any new values $$x_i$$, so all I have are the existing $$f_i$$. My $$x_i$$ in this case are in fact random, but in principle it should also work for $$x_i$$ on a dense regular grid.

• So you want $E\{f(X)\}$, where $X$ is normally distributed? It is just a simple integral, $\int f(x)g(x) dx$, where $g(x)$ is your normal density. What am I missing? Aug 3, 2021 at 0:15
• In the continuous limit, that equation looks right to me, but what do I do with that? Aug 3, 2021 at 2:30

On principle you simply cannot estimate$$\int_{-\infty}^{\infty} f(x) \varphi_\sigma(x)\,\text dx$$based on $$f(x_i)$$'s with $$x_i\in(-1,1)$$ since there is not information for $$f(x)$$ when $$x\not\in(-1,1)$$...
Now, if $$\sigma$$ is small enough for$$\int_{-1}^{1} \varphi_\sigma(x)\,\text dx\approx 1$$you can consider$$\frac{2}{n}\sum_{i=1}^nf(x_i)\varphi_\sigma(x_i)$$as a pseudo-importance sampler but, depending on the function $$f$$, the approximation can completely go south, i.e. be completely unreliable: take an extreme example when $$f$$ is very large on $$(-2,-1)$$ and very small or null over $$(-1,1)$$: $$f(x)=\begin{cases} 1/\varphi_\sigma(x) &\text{when }x\in(-2,-1)\\ 0 &\text{elsewhere} \end{cases}$$ Then $$\int_{-\infty}^{\infty} f(x) \varphi_\sigma(x)\,\text dx= 1$$ while $$\frac{1}{n}\sum_{i=1}^nf(x_i)\varphi_\sigma(x_i)=0$$
Obviously, if the support of $$f$$ is $$(-1,1)$$ then $$\int_{-\infty}^{\infty} f(x) \varphi_\sigma(x)\,\text dx= \int_{-1}^{1} 2f(x) \varphi_\sigma(x)\,\frac{\text dx}{2}=2\mathbb E[f(U)\varphi_\sigma(U)]$$when $$U\sim\mathcal U(-1,1)$$.
• Yes, I understand there could be a pathological $f(x)$. In this particular case, $f(x)$ can be considered 0 outside $[-1, 1]$ and also $\sigma$ is going to be sufficiently small so that $\int_{-1}^{1} \phi(x) dx \approx 0$. Just to make sure: $\phi_\sigma(x)$ the PDF for normal distribution, right? Then your answer would correspond to my initial intuition, except for the normalization with $\sum\phi_\sigma(x_i)$. Let me try that... Aug 3, 2021 at 10:25
• If $f(x)\equiv 1$, the average $\bar{f}$ should be 1 as well (independent of $\sigma$), which doesn't seem to be what I'm getting with $\frac{1}{n} \sum f_i \phi(x_i)$ (or with $\frac{2}{n} \sum f_i \phi(x_i)$ ) Aug 3, 2021 at 15:44
• Actually, I think it turns out that my original intuition was correct after all: $\left(\sum_{i=1}^{n} f_i \phi_\sigma(x_i)\right) / \sum_{i=1}^{n} \phi_\sigma(x_i)$ does seem to work! What was tripping me up was a complication that I left out of the description here: I actually have another "category" parameter in my data, and I want to do do the averaging for each category indpendently. If I do the averaging before a groupby for the category, I also have to divide by the number of categories. Or, do the groupby first. Aug 3, 2021 at 16:36