0
$\begingroup$

I am doing research on deep neural networks for prediction. I wanted to know what minimum size of dataset is required for training a deep network. Is there any limitation imposed on how much observations of predictior variables should be there?

$\endgroup$
  • $\begingroup$ this seems way too broad to be answered with any reasonable precision. training an autoencoder can be done with a few hundred frames, yet training an RL agent can easily take 50 million. $\endgroup$ – shimao May 20 '18 at 14:49
0
$\begingroup$

It largely depends on the problem. On high dimensional datasets like image dataset, it is not uncommon to need $10000$ training points to capture the nuances of data. For Natural Language Processing applications like Seq2Seq, I have trained models with just $3500$ data points.

If the application is more like a Data Analytics problem, where the number of features is typically not greater than $50$, around $1000$ points may be enough. But again, I emphasize that the number of data points needed largely depends on the problem.

One common problem that could occur if the dataset used is small is, if the Neural Network has high capacity (a large number of hidden units and/or a large number of layers), overfitting could occur because it is not hard for the Neural Network to memorize all the points. At the same time, if the dataset was large, the Neural Network would have been forced to capture the important variations only. There are lots of regularization methods and one must almost always use them. Check Dropout for example.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.