# Unbiased estimator and biased error

I'm having some trouble relating unbiased estimators and bias error. By bias error, I mean the bias error we talk about when analyzing "bias-variance tradeoffs." Is this bias error and an unbiased estimator related in any way?

I know that an unbiased estimator means that the expected value of the estimator is equal to the true value of the parameter being estimated.

For example, for linear regression, we have the linear model

$$y = X\beta + \epsilon$$ where $$\beta$$ is the true parameter that can't be observed. For least squares, we approximate this parameter with $$\hat{\beta} = (X^TX)^{-1}X^Ty$$

Under the Gauss-Markov theorem, we can show that $$E[\hat{\beta}] = \beta$$, indicating that the least squares estimator is unbiased. Does an unbiased estimator mean that the bias error is zero?

If so, then I am confused by this because the two seems to be separate concepts to me albeit both using the term "bias." Wikipedia defines bias error as "The bias error is an error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting)." Even though we have an unbiased estimator $$\hat{\beta}$$, we could still produce a model that doesn't accurately reflect the underlying data (for example using a linear model to approximate data that is non-linear).

• The confusion comes from different usages of the term "bias" from the precise "expectation of the estimator is the estimated quantity" to an equivalent for "error" and "misspecified". May 21, 2020 at 6:44
• @Xi'an Thanks. Just to clarify, the 2 usages of "bias" that I outlined in my OP are distinct from each other? May 21, 2020 at 16:36
• yes indeed..... May 21, 2020 at 17:17
• Jun 22, 2020 at 8:16
• @Xi'an I asked a similar question in stats.stackexchange.com/questions/473710/…. When the squared error is decomposed into a bias, variance, and noise term, an unbiased estimator does not mean the bias term in the decomposition of the error is zero, right? Jun 25, 2020 at 18:24

By definition, bias error is the difference between the expected value of the estimator and the true value being estimated. Therefore yes, when an estimator is unbiased it does directly imply that the bias error is 0. Directly from the wiki An estimator or decision rule with zero bias is called unbiased.