In neural networks that start with input layer, run through hidden layers, and ultimately reach the output layer, we start back-propagation from weights closer to output layer and go backward towards the input layer.
Now, in case of VAE, I was confused as to whether back-propagation start from the decoder all the way to encoder with an aim to maximize ELBO or is it something different?
log p(x) = ELBO + KLD between approx posterior and true posterior [Equation 1 of paper]
Since the KLD part of log evidence equation above (bold part) has true posterior which is intractable, in order to minimize it, we instead try to maximize ELBO.
Hence, we can say that maximizing ELBO is basically minimizing the KL divergence between approximate posterior and true posterior (i.e. q(z|x) and p(z|x)). We are maximizing ELBO so as to indirectly obtain the objective of minimizing the KLD between approximate posterior q(z|x) and true posterior p(z|x), which I feel is related to encoder as we are trying to find best approximate distribution of z given x.
So, my confusion and question is that since maximizing ELBO is, as explained above, associated with encoder, how does maximizing it helps to correctly adjust weights for the decoder?
Is it because of the following reason:
Although we are maximizing ELBO so as to indirectly minimize KLD between approximate posterior and true posterior, when ELBO is derived, we can see that it consists of reconstruction term along with the constraining KLD term (i.e. KLD between approximate posterior and prior). Since it consists of the reconstruction term, it is somehow connected to decoder as well. Hence, maximizing ELBO has impact on both encoder and decoder.
Additional Question: Is it ELBO that is taken derivative of with respect to weights when back propagating the VAE neural network?