I'm studying variational autoencoders and I cannot get my head around their cost function.
I understood the principle intuitively but not the math behind it: in the paragraph 'Cost Function' of the blog post here it is said:
In other words, we want to simultaneously tune these complementary parameters such that we maximize log(p(x|ϕ,θ)) - the log-likelihood across all datapoints xx under the current model settings, after marginalizing out the latent variables zz. This term is also known as the model evidence.
in other words we want to solve the log-likelihood function from the encoder/decoder parameters ϕ and θ in order to find the probability distribution
p from the input samples that better models them.
We can express this marginal likelihood as the sum of what we’ll call the variational or evidence lower bound LL and the Kullback-Leibler (KL) divergence DKLDKL between the approximate and true latent posteriors: log(p(x))=L(ϕ,θ;x)+DKL(qϕ(z|x)||pθ(z|x))
This is also just math from the definition of KL divergence as in the passages here.
for now the important thing is that it is non-negative by definition; consequently, the first term acts as a lower bound on the total. So, we maximize the lower bound LL as a (computationally-tractable) proxy for the total marginal likelihood of the data under the model.
This is also reasonable to me.
With some mathematical wrangling, we can decompose LL into the following objective function:
I tried to follow the 'mathematical wrangling' on the paper linked (2.2 - The variational bound) but couldn't obtain the function above.
Can somebody help me out in figuring it?