This might be a stupid question but this question is bugging me for a long time. When I first started working with neural networks we usually created a neural network which output vector of number values. We then used crossentropy or MSE loss for training classification and regression models respectively.
But now many generative models assumes that the output of neural network as a probability distribution and uses loss function such as Kullback Liebler (KL) Divergence, Jensen-Shannon divergence to train a model, example: GANs, VAEs etc.
Why in some training cases the outputs are assumed to be a probability distribution? What is the advantages and disadvantages of this assumption?