I'm studying variational inference (in the context of VAEs), and I'm watching this video at this time point. At this point in the video, the goal of approximating the intractable posterior $p_{\theta}(z|x)$ by an approximate posterior $q_{\phi}(z|x)$ has been set, and the ELBO formulation has been presented.
As is shown on the image, what the author of the video says at this point is that, since marginal log-likelihood will not change, maximizing ELBO minimizes the KL divergence between the $q$ and $p$ distributions (which is what we want). To quote more precisely, the author says : "We cannot compute [marginal log-likelihood] but we know that it will not change and so we have found a way around our KL divergence".
This sentence confused me. In the context of variational auto-encoders for generating data, I thought that the goal was specifically to increase the evidence by finding the optimal $\theta$ (and $\phi$) parameters, and so that evidence $p_{\theta}(x)$ will be maximized during the optimisation process (and I guess, $D_{KL}(q_{\phi}||p_{\theta})$ decreases also, but sort of as a by-product ?). So does evidence change or not during the training process ?
Side-question : I think I'm getting confused about some very basic statistical notations as they are sometimes inconsistent between different materials. Is it true that usually, a notation such as $p(X)$ is a distribution, and $p(x)$ is a number representing the probability $p(X=x)$ ? So evidence is a number, right, not a distribution ?