Why does the sign of the loss in VAEs appear to be backwards?

Question

I'm trying to fully understand Variational Autoencoders (VAEs) 1 and their math but one part keeps confusing me and I hope someone can give me in an intuitive explanation what I am missing.

Here is how I understood it and where I am getting confused:

The overall loss consists of 2 parts, the regularization loss and the reconstruction loss:

$$L_{vae}=L_{regularization}+L_{reconstruction}$$

The regularization loss ensures that our VAE encoder $q(z|x)$ produces latents that follow our prior distribution $p(z)$ (gaussian normal): $$L_{regularization} = KL(q(z|x) \parallel p(z))$$ if $q(z|x)$ is close to $p(z)$, the KL term is close to 0, hence, our goal is to minimize this term, because we want our latents to follow that prior distribution.

The reconstruction loss is computed based on maximizing the log-likelihood of our output given the latent variable z: $p(x|z)$. In other words, we try make sure that our output looks like our input, i.e. "reconstructing" it from our latents. And since maximizing the log-likelihood is the same as minimizing the negative log-likelihood, we can write:

$$L_{reconstruction} =-\mathbb{E}_{q(z|x)}[logp(x|z)]$$

From this follows:

$$L_{vae}= KL(q(z|x) \parallel p(z)) -\mathbb{E}_{q(z|x)}[logp(x|z)]$$

Now here is where I am confused:

In Kingma & Welling, "Auto-Encoding Variational Bayes" (2014) and everywhere else I am looking, the final loss is the above loss $*(-1)$:

$$L_{vae}= -KL(q(z|x) \parallel p(z)) +\mathbb{E}_{q(z|x)}[logp(x|z)]$$ or $$L_{vae}= \mathbb{E}_{q(z|x)}[logp(x|z)]-KL(q(z|x) \parallel p(z))$$

Main Question: Usually we do such a sign switch, since we want to convert our problem from a maximization to a minimization problem for optimizing it (e.g. with a neural network). But why are we doing it here? Are we not already "minimizing the negative log-likelihood" and is the KL term not already a minimization problem?

Answer 1

The question is starts from a mistaken premise. Equations $(1), (2), (3)$ in the article (Kingma & Welling, "Auto-Encoding Variational Bayes," 2014), refer to the likelihood, not the loss function. Likelihoods are maximized; maximizing $f(x)$ is equivalent to minimizing $-f(x)$. This quotation from the paper shows that these equations are likelihoods.

The marginal likelihood is composed of a sum over the marginal likelihoods of individual datapoints $\log p_\theta(x^{(1)}, \cdots, x^{(N)}) = \sum_{i=1}^N \log p_\theta(x^{(i)})$, which can each be rewritten as: \begin{align} \log p_\theta(x^{(i)}) = D_{KL}(q_\phi(z|x^{(i)})||p_\theta(z|x^{(i)})) + \mathcal{L}(\theta,\phi;x^{(i)}) \tag{1}\label{1} \end{align} The first RHS term is the KL divergence of the approximate from the true posterior. Since this KL-divergence is non-negative, the second RHS term $\mathcal{L}(\theta,\phi;x^{(i)})$ is called the (variational) lower bound on the marginal likelihood of datapoint $i$, and can be written as: \begin{align} \log p_\theta(x^{(i)}) \geq \mathcal{L}(\theta,\phi;x^{(i)}) &= \mathbb{E}_{q_\phi(z|x)}[- \log q_\phi(z|x) + \log p_\theta(x,z)] \tag{2}\label{2} \end{align} which can also be written as: \begin{align} \mathcal{L}(\theta,\phi;x^{(i)}) = - D_{KL}(q_\phi(z|x^{(i)}) || p_\theta(z)) + \mathbb{E}_{q_\phi(z|x^{(i)})} {\log p_\theta(x^{(i)} | z)} \tag{3}\label{3} \end{align}

(I've bolded the word “likelihood” in this quotation.)