Can increasing the training data reduce bias As per my understanding, there is high bias if the model is underfitted.
Does the number of records in training data affects bias?
I mean, if there is too less records in training data, can the model be called underfitted and have high bias?
Also, if there is high bias, can it be reduced by increasing the number of records in training data?
 A: In general, there's not necessarily any relationship between an estimator's bias and the sample size. However, estimators are often constructed so that their bias and variance decline as the sample size grows.
As a trivial counterexample, consider the case where you have some data $X_1, \dots, X_n$, and you'd like to estimate their mean. Now, a valid estimator is: $\hat\theta = 5$. That is, the estimator always returns $5$ no matter what the data look like, or how many data points you have.
A: Just to add more information to @Andrew 's comment.
First of all, bias and variance that you are talking about are properties of an estimator, not the data.
In this case, if my estimator is a static function which always returns 1. It's clear that getting more data doesn't reduce bias or variance.
But most of the time, we are talking about machine learning estimator which it's capable f learning given data. Its bias can be seen as a limitation of that model. So you cannot reduce the bias by adding more data -- but it might be reduced if you apply a transformation to the data to make it easier for a model to learn.
When people say that adding more data will decrease variance(not bias), as I understand, it is because that additional data reveal more information about the relationship that you are looking for so the model doesn't have to guess so it can be certain about the answer.
