# Convergence to gradient in limit of variance

I came across this equation in the original GAN paper (pg 2 https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf): $$\lim_{\sigma \rightarrow 0} \nabla_{\bf x} \mathbb{E}_{\epsilon \sim \mathcal{N}(0, \sigma^2 I)}[{f({\bf x} + \epsilon)}] = \nabla_{\bf x} f({\bf x}).$$

Is this a well known result, or easy to prove? Not sure how to proceed beyond the following:

\begin{align*} \lim_{\sigma \rightarrow 0} \nabla_{\bf x} \mathbb{E}_{\epsilon \sim \mathcal{N}(0, \sigma^2 I)}[{f({\bf x} + \epsilon)}] &= \lim_{\sigma \rightarrow 0} \int_{\epsilon} \nabla_{\bf x} f({\bf x} + \epsilon) p(\epsilon) d \epsilon \end{align*}.

Any tips or sources would be greatly appreciated.

I think this is true when $$f$$ has a continuous gradient, in which case $$\lim_{h \rightarrow \vec0} \nabla_x f(x+h) = \nabla_x f(x)$$ which is equivalent to $$\lim_{\sigma \rightarrow 0} \nabla_x f(x+\sigma z) = \nabla_x f(x)$$ for any value of $$z$$.
Then \begin{align}&\quad\ \lim_{\sigma \rightarrow 0} \nabla_x \mathbb{E}_\epsilon \left[f(x+\epsilon )\right] \\ &= \lim_{\sigma \rightarrow 0} \nabla_x \mathbb{E}_{z \sim \mathcal{N}(0,I)}\left[f(x + \sigma z)\right] \\ &= \mathbb{E}_{z \sim \mathcal{N}(0,I)}\left[\lim_{\sigma \rightarrow 0} \nabla_x f(x + \sigma z) \right] \\ &= \mathbb{E}_z \left[ \nabla_x f(x)\right] \\ & = \nabla_x f(x)\end{align}
• (+1) Of course, some further technicalities on $f$ are needed to formally exchange the limit and the expectation, but this is surely the level of technicality at which the GAN paper was thinking about it. Commented Feb 4, 2019 at 17:36