# Can increasing the training data reduce bias

As per my understanding, there is high bias if the model is underfitted.

Does the number of records in training data affects bias? I mean, if there is too less records in training data, can the model be called underfitted and have high bias?

Also, if there is high bias, can it be reduced by increasing the number of records in training data?

In general, there's not necessarily any relationship between an estimator's bias and the sample size. However, estimators are often constructed so that their bias and variance decline as the sample size grows.

As a trivial counterexample, consider the case where you have some data $$X_1, \dots, X_n$$, and you'd like to estimate their mean. Now, a valid estimator is: $$\hat\theta = 5$$. That is, the estimator always returns $$5$$ no matter what the data look like, or how many data points you have.

• So when we say that "estimators are often constructed so that their bias and variance decline as sample size grows" Does it mean that there is a possibility of bias getting reduced when sample size is increased? Commented Jan 10, 2020 at 17:03
• @AnkitD It depends on your model. Different models will have different theoretical properties. Commented Jan 12, 2020 at 13:35