0
$\begingroup$

Below is some steps for differentiating a function wrt a set of parameters $\phi$ using the "reparameterisation trick" (Kingma & Welling 2013).

However after applying the derivative as follows I cannot follow how the underlined portion of the derivative came about. Could someone simplify or explain the origin of this term?

test

$\endgroup$
  • $\begingroup$ Do you understand where the second term on that line comes from? $\endgroup$ – jbowman Mar 11 at 14:43
  • $\begingroup$ Yes. That is chain rule. To be honest I only would have ended up with the second term and not included the first term $\endgroup$ – pche8701 Mar 11 at 22:21
1
$\begingroup$

While you are right on the second term and the use of the chain rule, your distribution $q$ also directly depends on $\phi$ as made explicit by the subscript $q_\mathbf{\phi}$. That's why I prefer the notation $q(\cdot \, ;\phi)$, everything is more clear.

Denoting from the beginning: $\theta = f(\epsilon \, ; \phi)$ and $g = \log q(\theta \, ; \phi)$, we have using Leibniz's notation:

\begin{align*} \frac{\partial g(\phi, \theta)}{\partial \phi} = \left. \frac{\partial g(\phi, \theta)}{\partial \phi} \right|_{\theta = f(\epsilon \, ; \phi)} + \frac{\partial g(\phi, \theta)}{\partial \theta} \frac{\partial f(\epsilon \, ; \phi)}{\partial \phi} \; \textrm{,} \end{align*}

where the derivative of the first term on the right-hand side is computed with $\theta$ fixed

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.