In machine learning, I've heard a lot of references to the "Bayesian Approach" or a "Bayesian Model." I found this question and it seems to suggest that a Bayesian Model, very broadly speaking, attempts to estimate a posterior probability distribution from a data set, given a prior. But isn't that what machine learning is in general, trying to estimate some distribution function given data? Maybe I'm just not understanding what a Bayesian model really is, but, doesn't that imply that all models that return a probability distribution (i.e. your vanilla Logistic Regression model) would fall under this "Bayesian" category? I somehow don't think that's true.
In any case then, do Bayesian Models offer some sort of inherent advantages or disadvantages over non-Bayesian models? I realize that's a very much dependent on the context of the question, but I'm asking if the model's Bayesian quality inherently imparts different properties on the model, regardless of the specifics of the model outside of it being Bayesian?