6
$\begingroup$

I know that bias is the difference between this estimator's expected value and the true value of the parameter being estimated. In classical approach the parameter has one particular true value, meanwhile in Bayesian approach the parameter value (is fixed but) is chosen from a probability distribution.

But is it Bayesian or frequentist concept?

Improve this question
$\endgroup$
5
  • $\begingroup$ Maybe it is a statistical concept? (But it seems to be difficult to formalize in a bayesian context). $\endgroup$
    – kjetil b halvorsen ♦
    Commented May 24, 2017 at 14:45
  • $\begingroup$ I do not think Bayesian approach has the definition of bias. Next question is: how does the Bayesian evaluate the estimators given no definition of bias? because bias and MSE cannot be used. $\endgroup$
    – user158565
    Commented May 24, 2017 at 16:43
  • $\begingroup$ @a_statistician E.T. Jaynes, who was quite the militant Bayesian, seems to use the standard frequentist definition of bias in his writings. He also uses MSE as the default way to evaluate estimators, unless some problem-specific criterion is known. Jaynes's main quarrel is that reducing bias does not necessary lead to reduced MSE, so he calls it an 'emotionally loaded term', and that 'orthodoxians... are caught in psychosemantic trap of their own making'... $\endgroup$
    – juod
    Commented May 24, 2017 at 17:50
  • $\begingroup$ @juod The problem is the definition of bias in Bayesian. I said NO, and you said Jaynes had one. Could you introduce Jaynes's bias? Then we can discuss Jaynes's bias. $\endgroup$
    – user158565
    Commented May 24, 2017 at 18:54
  • $\begingroup$ @a_statistician it seems to me that Jaynes is fine with the concept of one true parameter value, because he introduces bias as $E(\hat\theta) = \theta$, and proceeds to argue only about whether it should be the criterion for estimator choice. But I could certainly be wrong. $\endgroup$
    – juod
    Commented May 24, 2017 at 19:54

1 Answer 1

Reset to default
3
$\begingroup$

Suppose there is a model for the data $Y$ that depends on a parameter $\theta$ and, for a particular experiment, there is a true value of the parameter, $\theta_0$. You develop an estimator $\hat\theta = \hat\theta(Y)$, i.e. the estimator is a function of the data $Y$. Then the bias is $$ bias(\hat\theta) = E_{Y|\theta_0}[\hat\theta(Y) - \theta_0] $$ where the expectation is taken with respect to the randomness of the data $Y$ for the given true value of the parameter $\theta_0$ (and the subscript on the expectation attempts to make this explicit). As we are talking about an expectation over possible realizations of data, this is a frequentist concept.

In the description above, I have not mentioned how the estimator arises. This estimator could be a method of moments, maximum likelihood, Bayes, or something else estimator. Thus, the concept of bias of an estimator is frequentist, but the estimator itself could arise from a Bayesian analysis.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ This is still ambiguous. The whole idea of there being a true, fixed theta is itself very non-Bayesian. For strict Bayesian, the parameter itself is random. All we have is prior uncertainty about theta, and posterior uncertainty about theta. If you don't have a true, fixed theta, you can't compute the bias as given above. You need to average that bias over the distribution of theta itself. $\endgroup$
    – AOGSTA
    Commented Mar 10, 2019 at 23:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.