0
$\begingroup$

Have you ever seen anything like a theoretical approach for determining the optimal size (or perhaps some bounds for it) of the training set for a neural network?

I know that this is a very broad question, since the optimality should be considered with respect to a specific loss function; and there are several kinds of neural networks. Please feel free to consider the specific cases which you know about, if necessary - although I am particularly interested in perceptrons with 1 hidden layer for classification, reducing the cross-entropy loss function.-

I would like to find something more rigorous than an empirical comparison of the classification errors over training and validation data for different sizes of the training set.

$\endgroup$
  • $\begingroup$ Are we assuming that the variables are Independent and identically distributed random variables. And are the training examples are independent and well distributed? $\endgroup$ – show_stopper Mar 19 at 20:08
  • $\begingroup$ @show_stopper yes, you may assume all that $\endgroup$ – Javi Mar 19 at 20:09
0
$\begingroup$

Have you ever seen anything like a theoretical approach for determining the optimal size (or perhaps some bounds for it) of the training set for a neural network?

What you are referring to is the topic of computational learning theory. One way to measure it is to use Vapnik-Chervonenkis dimension.

There are some results on VC dimension bounds of neural networks: Vapnik-Chervonenkis Dimension of Neural Nets

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.