I realize the methodology pursued by the Frequentist and Bayesian camps generally differ. However, one method of estimation that they do share is optimization of a certain function:
- Frequentists maximize the likelihood function, giving the Max. Likelihood (ML) estimator.
- Bayesians maximize the posterior function, giving the Max A-Posteriori (MAP) estimator.
Both functions will typically have been constructed using Baye's rule/theorem, which is universally agreed upon, and which might have been applied once (in "batch mode") or multiple times iteratively.
Similarly, both Frequentists and Bayesians will deduce their interval (confidence/credibility) from this function.
So if the prior is uninformative (assuming we can formulate such a prior), there should be no distinction between the "results" obtained by Bayesians and Frequentists, even though the interpretation of said results will be different.
If this is right, then the only practical difference between Bayesians and Frequentists is the prior. Is this true?
Actually, the optimization bit of my question is a bit misleading, as it is only a specific example of differences between Bayesian and Frequentist thinking. My question could be posed simply as the difference simply between the likelihood function and the posterior. For example, would frequentists ever use MCMC to calculate the likelihood function?
Edit, 10 years on: This was a confused question from a confused student. To be fair, the topic tends to confuse the uninitiated. Thanks to everyone who participated in the discussions.