1
$\begingroup$

So according to most of the sources I saw, increasing training data size will only benefit high-variance ML models by exposing the model to less spurious patterns, which occur more frequently in smaller data sets.

However, I observed that my NLP model had consistently low (which suggests that it is not due to high variance) prediction accuracy when the training data set is small, and its prediction accuracy improved dramatically with more training data.

The image below is the learning curves for three runs of the same NLP algorithm. Learning curves for three runs of the same NLP algorithm.

This seems intuitive as statistical methods need more data to be trained properly. However, I'm interested to know if there's any theory/rules that explains this observation?

$\endgroup$
2
$\begingroup$

However, I observed that my NLP model had consistently low (which suggests that it is not due to high variance)...

I don't believe this is necessarily the case. Variance in the sense of bias-variance tradeoff refers to the variance over different training sets of the model's output on a fixed (unseen) datapoint. A very high-variance model will likely have very poor scores on any new test set.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.