I have some convolutional neural network and I want to know how to get the data set size required for training?
How to get the data set size if I want to train a perceptron? Which math tool can be used to calculate the estimate?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
There's really no fixed rule that you can apply here. The number of training samples for training depends on the nature of the problem, the number of features, and the complexity of your network architecture. Try "simple" architectures first, i.e., fewer layers, fewer units per layer and experiment a bit with different training sizes and architectures to get a feeling for that. I know, the answer may be a bit disappointing, but as far as I know, it's all empirical for now.
Also, maybe learning curves could help (although, be aware that it's expensive; it's useful for developing a "feeling" for the dataset and model complexity though) E.g., I did this one for a MNIST subset using a simple softmax algorithm (1-layer) some time ago. I used 1500 samples for testing for the different training set sizes, and I would conclude from this figure that more training data may help to fit a "more accurate" model.
I'll copy my answer from the very related question How few training examples is too few when training a neural network? (any update will be performed there):
It really depends on your dataset, and network architecture. One rule of thumb I have read (e.g., in (2)) was a few thousand samples per class for the neural network to start to perform very well.
In practice, people try and see. It's not rare to find studies showing decent results with a training set smaller than 1000 samples.
(2) Cireşan, Dan C., Ueli Meier, and Jürgen Schmidhuber. "Transfer learning for Latin and Chinese characters with deep neural networks." In The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1-6. IEEE, 2012. https://scholar.google.com/scholar?cluster=7452424507909578812&hl=en&as_sdt=0,22 ; http://people.idsia.ch/~ciresan/data/ijcnn2012_v9.pdf:
For classification tasks with a few thousand samples per class, the benefit of (unsupervised or supervised) pretraining is not easy to demonstrate.
The "data set size" is property of the data set, not of the NN. If you are working with MNIST data set - the full data set is 60,000 images. If you split 10% for validation, you'd have 54,000 images for training. The training data set size will be 54,000.