Convergence to gradient in limit of variance

Question

I came across this equation in the original GAN paper (pg 2 https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf): $$\lim_{\sigma \rightarrow 0} \nabla_{\bf x} \mathbb{E}_{\epsilon \sim \mathcal{N}(0, \sigma^2 I)}[{f({\bf x} + \epsilon)}] = \nabla_{\bf x} f({\bf x}).$$

Is this a well known result, or easy to prove? Not sure how to proceed beyond the following:

$$\begin{align*} \lim_{\sigma \rightarrow 0} \nabla_{\bf x} \mathbb{E}_{\epsilon \sim \mathcal{N}(0, \sigma^2 I)}[{f({\bf x} + \epsilon)}] &= \lim_{\sigma \rightarrow 0} \int_{\epsilon} \nabla_{\bf x} f({\bf x} + \epsilon) p(\epsilon) d \epsilon \end{align*}. $$

Any tips or sources would be greatly appreciated.

Answer 1

I think this is true when $f$ has a continuous gradient, in which case $$\lim_{h \rightarrow \vec0} \nabla_x f(x+h) = \nabla_x f(x)$$ which is equivalent to $$\lim_{\sigma \rightarrow 0} \nabla_x f(x+\sigma z) = \nabla_x f(x)$$ for any value of $z$.

Then $$\begin{align}&\quad\ \lim_{\sigma \rightarrow 0} \nabla_x \mathbb{E}_\epsilon \left[f(x+\epsilon )\right] \\ &= \lim_{\sigma \rightarrow 0} \nabla_x \mathbb{E}_{z \sim \mathcal{N}(0,I)}\left[f(x + \sigma z)\right] \\ &= \mathbb{E}_{z \sim \mathcal{N}(0,I)}\left[\lim_{\sigma \rightarrow 0} \nabla_x f(x + \sigma z) \right] \\ &= \mathbb{E}_z \left[ \nabla_x f(x)\right] \\ & = \nabla_x f(x)\end{align}$$