1
$\begingroup$

Is it correct to compare Bayesian and Classical estimates using Mean Squared Error (MSE)? MSE is a criterion that is used in the classical paradigm.

For example: I am comparing the performance of classical and Bayesian estimators for a distribution, in classical, it is MLE and Bayesian paradigm it is posterior mean (using squared error loss function).

Now, is it correct to compare MLE and Posterior mean on the basis of MSE. The confusion here is that MSE is used in classical paradigm. So, can we compare two paradigm based on the criteria of one of them.

$\endgroup$
3
  • $\begingroup$ I think you may need to edit this with more details. When you say you want to compare them on what dimension would that be? $\endgroup$
    – mdewey
    Mar 21, 2021 at 17:03
  • $\begingroup$ are you trying to compare models estimates on data? $\endgroup$
    – Good Luck
    Mar 22, 2021 at 14:37
  • 1
    $\begingroup$ I am comparing estimators for a specific distribution. MSEs are calculated through simulation. $\endgroup$
    – rgt
    Mar 22, 2021 at 14:44

2 Answers 2

2
$\begingroup$

From what I understand, your query concerns whether it is "correct" to compare the Bayes estimators, belonging to a "Bayesian paradigm", with other frequentist estimators such as maximum likelihood or method of moments estimators, which belong to a "frequentist paradigm". And that this is essentially a question of whether these estimators, generated under different paradigms, are commensurable according to the metric of mean squared error.

To the best of my understanding, one can compute Bayes estimators without necessarily adopting Bayesian semantics nor the epistemology - in this case, the frequentist would view Bayes estimators purely as an algorithmic procedure for generating a point estimator. That is, after setting up the Bayesian machinery of a posterior, and computing the posterior mean, you then retain the subsequent estimator $\hat{\theta}(x) = \mathbb{E}[\theta | X = x]$, and bin the semantics and interpretation - $\hat{\theta}$ is now just one of many possible point estimators.

On this basis, I see no reason why two competing point estimators cannot be compared on the basis of mean squared error.

However, if you want to retain the Bayesian semantics and epistemology of the Bayes estimator, that is, to treat the parameter $\theta$ not as fixed unknown number, but a random variable whose uncertainty is captured by a distribution, and say compare it to a maximum likelihood estimator $\tilde{\theta}$ by comparing the MSE of $\hat{\theta}$ and $\tilde{\theta}$, then I think there might be good reasons to question the use of MSE. In that it is a frequentist metric whereby the "quality" of an estimator is measured by its proximity to a fixed, "true", unknown parameter $\theta$.

However, on this latter point I am unsure, and would be keen to hear from a seasoned statistician about this.


As a side-note, in the contexts in which I've encountered theoretical statistics, that is, in statistical machine learning, metrics such as mean-squared error are only a preliminary stepping stone for evaluating the quality of estimators. I cannot speak for theoretical statistics, but I know that in statistical machine learning, there is much more of a focus on using the more formal framework of minimax theory to evaluate the quality of estimators.

$\endgroup$
0
$\begingroup$

Is it correct to compare Bayesian and Classical estimates using Mean Squared Error

Yes. This is possible, but there is difference in approach.

Frequentist methods calculate MSE numerically without respect to any prior beliefs. This is how they achieve state of the art in many areas (i.e. linear regression and lasso regression), and Bayeisan methods need a prior belief as a reference point of aspect to start.

Bayesian methods are complementary to frequentist, and you may mix both methods to get Empirical Bayes methods.

At the end MSE will be MSE whatever approach you take. The difference:

  • in classical approach you get a number (point estimate)
  • in Bayesian approach you get a distribution of likely values and you need to deal with point estimate as prior belief. This prior can be: "frequentist" MSE, or mean or mode of the posterior distribution.

Bayesian method is natural approach to infrequence and Bayes formula is a mathematical model of the inference. At the same time this same formula is how we learn new improved parameters by updating the prior parameter belief with the posterior parameter belief. In other words bayesian approach is belief shifting system that produces the update of parameters (read: learns).

Both approaches share the same statistical primitives and there should be no terminology difference. Say, likelihood is the same for both approaches and there should be no difference in any respect. But I would like to hear comments on the last.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.