# Are the bias term b and the Bias in "Bias–variance tradeoff" the same thing?

In ADALINE algorithm , with $y=x*w+b$, where $x$ is the feature vector of a sample and $w$ is the weight vector, the update rule (SGD) for the bias $b$ is: $b \leftarrow b + \eta(o - y)$.

With Gradient Descent, the final $b$ is actually: $$b \leftarrow b_{init} + \eta\sum_{i=1}^n(o_i - y_i)$$ where $b_{init}$ is a random initial value for $b$.

On the other hand, according to Bias–variance tradeoff, the mean square error decomposition is: $$Err(x)=Bias^2+Variance+Irreducible\ Error$$, where $Bias$ is: $$Bias=E[\hat{f}(x)]-f(x)$$

• What is the relation between $b$ and $Bias$? Are they the same thing?
• Regarding the equation calculation, is $E[\hat{f}(x)]-f(x)$ the sum of errors like $b$?

In the first example, bias refers to the intercept of a linear model. It is simply a parameter just like $w$ that needs to be learned.
• So it seems they don't have any relation except they both are called bias. For the second $Bias$, do people actually calculate/analyze it for a certain purpose? Or it is simply an abstract concept for understanding the $Bias VS. Variance$? Jun 18, 2018 at 14:47