0
$\begingroup$

What are some popular solutions in dealing with very little training data? Do these solutions rely on generating more data (e.g. bootstrapping, SMOTE, etc.)? Or do they rely on methods that do not really need a lot of training data (e.g. ensemble methods like random forests, etc.)?

$\endgroup$
0
$\begingroup$

A separate hold-out set for testing might be reasonable when there are thousands of cases to examine. Even then, testing on a hold-out from a single data set does not serve the same purpose as testing on an independent data set. See this page for one introduction to those issues.

So in practice there often are "very little training data," as you put it. You don't really "generate more data" with bootstrapping or SMOTE, you simply use the data that you have in efficient ways to try to produce models that avoid overfitting and provide a reasonable chance of dealing well with new data. Random forests inherently use bootstrapping in this way. And don't forget that this site itself is named "Cross Validated," one frequently used approach to overcome the issue of "very little training data."

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.