Are the bias term b and the Bias in "Bias–variance tradeoff" the same thing?

Question

In ADALINE algorithm , with $y=x*w+b$, where $x$ is the feature vector of a sample and $w$ is the weight vector, the update rule (SGD) for the bias $b$ is: $b \leftarrow b + \eta(o - y)$.

With Gradient Descent, the final $b$ is actually: $$b \leftarrow b_{init} + \eta\sum_{i=1}^n(o_i - y_i)$$ where $b_{init}$ is a random initial value for $b$.

On the other hand, according to Bias–variance tradeoff, the mean square error decomposition is: $$Err(x)=Bias^2+Variance+Irreducible\ Error$$, where $Bias$ is: $$Bias=E[\hat{f}(x)]-f(x)$$

• What is the relation between $b$ and $Bias$? Are they the same thing?
• Regarding the equation calculation, is $E[\hat{f}(x)]-f(x)$ the sum of errors like $b$?

Answer 1

No, they are not the same thing.

In the first example, bias refers to the intercept of a linear model. It is simply a parameter just like $w$ that needs to be learned.

In the second example, bias is the difference between the ground truth and the expected prediction of the model. Here, the expectation is taken over different training sets from the same ground truth data distribution. It expresses how much is the specific model going to be wrong on average, thus the name bias.