Understanding reparameterization trick and training process in variational autoencoders

I am trying to understand variational autoencoders, particularly the sampling component and the reparameterization trick. I understand that instead of using a fixed determinstic latent representation as in traditional autoencoders, variational autoencoders involve computing mean and standard deviation vectors. These vectors are then used to sample latent vectors, which in turn can generate new data.

However, I am trying to understand where sampling fits in the training process, if at all? My understanding is that the stochastic aspect is only relevant for generating new data (after training is complete), but not for training the encoder/decoder networks. Is that correct? Or does sampling also occur during training the variational autoencoder model, and if so how does this work?

For a Gaussian VAE, each input is encoded to 2 vectors, one for means and one for standard deviations. Suppose your latent representation is 2 dimensional. You have a mean vector $$\mu = \begin{bmatrix}\mu_1 \\ \mu_2 \end{bmatrix}$$ and covariance $$\Sigma = \begin{bmatrix}\sigma_1^2 & 0 \\ 0 & \sigma_2^2 \end{bmatrix}$$. Then you use the re-parameterization trick to draw a random vector from this distribution. Then the decoder reconstructs the data.