How to obtain the functional derivative in variational inference? Referring to David Blei's notes on variational inference, I wonder how to get the derivative of $q(z_k)$, the distribution of $z_k$, in eq. 23 from eq. 22. 
Namely, eq. 22 is
$L_k = \int q(z_k) \mathbb{E}_{-k}[\log p(z_k | z_{-k},x)]dz_k - \int q(z_k)\log q(z_k) dz_k$ and,
eq. 23 is
$\frac{dL_k}{dq(z_k)} = \mathbb{E}_{-k}[\log p(z_k | z_{-k},x)] - \log q(z_k) - 1$
I suppose the derivation uses functional derivatives, which I know little about. I'd very appreciate a detailed derivation. 
 A: Let's streamline the notation by fixing a function $f$ and considering a functional
$$\mathcal{L}[q] = \int (q(z) f(z) - q(z) \log(q(z))) dz.$$
A variation $h$ is a function for which $q+h$ is still the same kind of function as $q$ (e.g., continuous or non-negative or whatever you need).  The effect of changing $q$ to $q+h$ is found in the usual way: we compare the results of $\mathcal{L}$ by subtracting the original value from the new one:
$$\eqalign{
\frac{\delta\mathcal{L}}{\delta q}[h] &= \mathcal{L}[q+h] - \mathcal{L}[h] \\
&= \int \left((q+h)(z) f(z) - (q+h)(z) \log((q+h)(z)) - \left(q(z) f(z) - q(z) \log(q(z))\right)\right) dz.}$$
Let's now restrict attention to variations $h$ that have infinitesimally small values so that we may be free to neglect second-order terms.  Specifically, that means we plan to use a product-like differentiation rule to approximate the product difference
$$(q+h)(z)\log((q+h)(z)) - q(z)\log(q(z)) \approx h(z)\log(q(z)) + q(z) \frac{h(z)}{q(z)}.\tag{1}$$
To see where this came from, compare it to taking the differential of the function (not functional)
$$d(x \log(x)) = (dx) \log(x) + x \left(\frac{dx}{x}\right).$$
Plugging $(1)$ into $\frac{\delta\mathcal{L}}{\delta q}[h]$, simplifying, and factoring out the common factors of $h(z)$ yields
$$\frac{\delta\mathcal{L}}{\delta q}[h] = \int\left(f(z) - \log(q(z)) - 1\right) h(z) dz.$$
If this is to vanish (to second order in $h$, anyway) for all infinitesimal $h$, then--assuming that at each $z$ in the domain of integration there exist some variations $h$ that are nonzero in some neighborhood of $z$--the right hand side can vanish only when the coefficient of $h$ itself vanishes for all $z$; that is,
$$f(z) - \log(q(z)) - 1 = 0.$$
That's Equation 23 in the referenced notes.  It means the functional $\mathcal{L}$ is stationary at $q$.  That's a necessary (but not sufficient) condition for $\mathcal{L}$ to have a local extremum at $q$.
