# Bayes by backprop unbiased monte carlo gradients

I am currently trying to understand a paper on bayesian neural networks whereby the authors use a bayes by backprop approach to learn weight uncertainties in the neural networks.

I am trying to understand the derivation for proposition 1 in the paper. Particularly, I am not sure how

$$\frac{\partial}{\partial\theta}\int f(\boldsymbol{w},\theta)q(\epsilon)d\epsilon = E_{q(\epsilon)}[\frac{\partial f(w,\theta)}{\partial w} \frac{\partial w}{\partial \theta} + \frac{\partial f(w,\theta)}{\partial \theta}]$$

I am not sure why there is an additional $$\frac{\partial f(w,\theta)}{\partial \theta}$$ inside the expectation ? since I thought $$\frac{\partial f(w,\theta)}{\partial w} \frac{\partial w}{\partial \theta} = \frac{\partial f(w,\theta)}{\partial w}$$

• isn't the first term inside the expectation quantity on the left hand side equal to the right hand side term by chain rule? Dec 7, 2020 at 1:36

Okay I think this is why there is an additional term. $$f$$ is a function of $$\textbf{w}$$ and $$\theta$$. Since $$\textbf{w}$$ is composed of $$\theta$$, we apply chain rule to obtain $$\frac{\partial f}{\partial w}\frac{\partial w}{\partial \theta}$$. Since $$f$$ is also a function of $$\theta$$. we also differentiate with respect to $$\theta$$, yielding $$\frac{\partial f}{\partial \theta}$$