Are all generative Models based on Bayes?

Question

Reading about deep learning I encounter various different kinds of hierarchical networks, many of which are generative.

1) Are all of the generative networks based on Bayes? 2) If not, how do they form the model that enables them to be generative?

Please explain in layman's English since my math is very limited.

I have read the excellent discussion at The connection between Bayesian statistics and generative modeling, but if I understand it correctly, it does not answer my question. It says there is a close relationship between Bayes and generative models but not that all generative models are based on Bayes. Nor does it say, if they are not, how they form their model.

Thank you.

Answer 1

A generative model is basically a story of how the data could have been generated.

Inside this story, we leave some parameters unspecified: these are the parameters that we are interested in learning about by collecting the data.

For example, if we have collected the height of 100 persons, and we want to determine the mean height in the county these people come from, one fiction we could come up with is: "some person's height = a Gaussian centered a the population mean with std 30 cm"

Now, once we have a generative model, we can ask what the data tells us about the parameters of interest, under this model. You can be Bayesian here: you would then compute the posterior of the paramater of interest. But you don't have to be: you can also, for example, find the parameter-value which fits the data the best: the maximum-likelihood value. That's a frequentist approach to generative models.

So the answer to your question is: "no". Generative models are necessary to apply bayesian methods, but you can also do maximum-likelihood (frequentist) inference on them.