Why do variational autoencoders not find the "best" latent variables? From my understanding:
Variational autoencoders sample the latent variables $y$ using a proposal distribution $q$ of the observed variables $x$.  The objective is that the decoder $p$ applied to $y$ defines a distribution whose value at $x$ is supposed to be maximized (reconstruction cost) plus a regularizer term.
Why not forget about $q$, and for every example $x$, iteratively update $y$ so as to maximize the objective?  That is, setting $y$ to
$$\arg \max_y P(x | y)$$
rather than sampling it from $q(y \mid x)$.
Is $q$ an approximation motivated by inference speed?
 A: Typically, we use the prior $p(y) = \mathcal{N}(0, I)$ and choose $p(x|y) = \mathcal{N}(\mu, \sigma = f(y;\theta))$ for some neural network $f(\cdot; \theta)$. The ELBO (evidence lower bound) is the objective of the VAE:
$$\log p(x) \geq E_{y \sim q(y|x)}[\log p(x|y)] - \mathcal{D}_{\text{KL}}(q(y|x)||p(y))$$

iteratively update $y$ so as to maximize the objective?

I'm not completely sure what you mean by "update $y$". There's no single $y$ value in our model to update, since it is a latent variable. We can only update our variational approximation of the posterior distribution of $y$ -- which we do by learning the parameters of the model of $q$.

Is q an approximation motivated by inference speed?

Yes, while training, we would like to sample $y$ from the true posterior distribution $p(y|x) = \frac{1}{Z} p(y)p(x|y)$ -- this would eliminate the gap between the ELBO and the true evidence $\log p(x)$ -- the inequality would become an equality. Unfortunately, sampling from this distribution is computationally intractable, so we resort to an approximation.
