Let us suppose that I have data that is normally distributed. I wish to find the $\mu$ and $\sigma$ parameters. The common sense thing to do is to simple calculate the sample mean and deviation. No fancy math, be done with it, and understand that a margin-of-error is expected. If we wish to quantify the margin of error we can use a confidence interval.
However, the fancy Bayesian approach is instead provide probability distributions for the two parameters. What benefit does this bring? I imagine there are problems were Bayesian approaches are better, but in this simplistic example that was mentioned what is the benefit?