I have a small data set with about 400 observations and 5 features. I want to train a neural network using the data. I use cross-validation and early stopping. I suspect that the data set size is too small. However, it's just a feeling.

What are the symptoms of having too few training samples? How can I check if the amount of data is sufficient?

You might argue that a bad model fit is a symptom but how do I know, if the model fits the data bad? Is a RMSE of 0.6 bad? Is a R2 of 0.4 bad?


As a rule of thumb, make sure that number of training Samples is $\geq$ 10*(VC-dimension of the model)

VC dimension ~ number of effective tunable parameters $\leq$ total number of weights in your neural net

Related :- How few training examples is too few when training a neural network?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.