# Is bias a frequentist concept or a Bayesian concept?

I know that bias is the difference between this estimator's expected value and the true value of the parameter being estimated. In classical approach the parameter has one particular true value, meanwhile in Bayesian approach the parameter value (is fixed but) is chosen from a probability distribution.

But is it Bayesian or frequentist concept?

• Maybe it is a statistical concept? (But it seems to be difficult to formalize in a bayesian context). – kjetil b halvorsen May 24 '17 at 14:45
• I do not think Bayesian approach has the definition of bias. Next question is: how does the Bayesian evaluate the estimators given no definition of bias? because bias and MSE cannot be used. – user158565 May 24 '17 at 16:43
• @a_statistician E.T. Jaynes, who was quite the militant Bayesian, seems to use the standard frequentist definition of bias in his writings. He also uses MSE as the default way to evaluate estimators, unless some problem-specific criterion is known. Jaynes's main quarrel is that reducing bias does not necessary lead to reduced MSE, so he calls it an 'emotionally loaded term', and that 'orthodoxians... are caught in psychosemantic trap of their own making'... – juod May 24 '17 at 17:50
• @juod The problem is the definition of bias in Bayesian. I said NO, and you said Jaynes had one. Could you introduce Jaynes's bias? Then we can discuss Jaynes's bias. – user158565 May 24 '17 at 18:54
• @a_statistician it seems to me that Jaynes is fine with the concept of one true parameter value, because he introduces bias as $E(\hat\theta) = \theta$, and proceeds to argue only about whether it should be the criterion for estimator choice. But I could certainly be wrong. – juod May 24 '17 at 19:54

Suppose there is a model for the data $Y$ that depends on a parameter $\theta$ and, for a particular experiment, there is a true value of the parameter, $\theta_0$. You develop an estimator $\hat\theta = \hat\theta(Y)$, i.e. the estimator is a function of the data $Y$. Then the bias is $$bias(\hat\theta) = E_{Y|\theta_0}[\hat\theta(Y) - \theta_0]$$ where the expectation is taken with respect to the randomness of the data $Y$ for the given true value of the parameter $\theta_0$ (and the subscript on the expectation attempts to make this explicit). As we are talking about an expectation over possible realizations of data, this is a frequentist concept.