1
$\begingroup$

I see in this document on variational inference that (p.5):

$L_k = \int q(z_k)E_{-k}[\log(p|z_{-k},x)]dz_k - \int q(z_k)\log q(z_k)dz_k$

It is stated that taking the derivative with respect to $q(z_k)$ results in:

$\frac{dL_k}{dq(z_k)}=E_{-k}[\log(p|z_{-k},x)] - \log q(z_k) - 1$

It looks to me like it has been argued that:

$\frac{d}{dq(z_k)}\int q(z_k)E_{-k}[\log(p|z_{-k},x)]dz_k = \frac{d}{dq(z_k)}[q(z_k)E_{-k}[\log(p|z_{-k},x)]] = E_{-k}[\log(p|z_{-k},x)]$

and

$\frac{d}{dq(z_k)}[- \int q(z_k)\log q(z_k)dz_k]=-\frac{d}{dq(z_k)}[q(z_k)\log q(z_k)]= -\log q(z_k) - 1 $

I can't see why this would work - it looks like there's been a bonus differentiation by $dz_k$. Can someone fill me in on why this is valid?

$\endgroup$
2
$\begingroup$

It works because these are functional derivatives.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.