I'm trying to explain to a nontechnical colleague of mine what a Bayesian approach is. I realized that despite having used Bayesian methods on more than one occasion in the past, I don't have an intuitive definition of what makes an approach Bayesian or not.
Based on several definitions I've seen in textbooks and online resources, the term "Bayesian" seems to mean:
Choosing a prior model, and then updating this prior model with new empirical data to obtain an improved posterior model.
This comes from applying Bayes rule to the context of modeling:
$P(Model\: |\: Data) \propto P(Data\: |\: Model) \:P(Model)$
But then isn't this just the definition of supervised machine learning in general?
What makes an approach specifically Bayesian and not just supervised learning? Or is it the case that all supervised learning really boils down to an application of Bayes rule?