0
$\begingroup$

Have you ever seen anything like a theoretical approach for determining the optimal size (or perhaps some bounds for it) of the training set for a neural network?

I know that this is a very broad question, since the optimality should be considered with respect to a specific loss function; and there are several kinds of neural networks. Please feel free to consider the specific cases which you know about, if necessary - although I am particularly interested in perceptrons with 1 hidden layer for classification, reducing the cross-entropy loss function.-

I would like to find something more rigorous than an empirical comparison of the classification errors over training and validation data for different sizes of the training set.

Improve this question
$\endgroup$
2
  • $\begingroup$ Are we assuming that the variables are Independent and identically distributed random variables. And are the training examples are independent and well distributed? $\endgroup$
    – show_stopper
    Commented Mar 19, 2019 at 20:08
  • $\begingroup$ @show_stopper yes, you may assume all that $\endgroup$
    – Javi
    Commented Mar 19, 2019 at 20:09

1 Answer 1

Reset to default
0
$\begingroup$

Have you ever seen anything like a theoretical approach for determining the optimal size (or perhaps some bounds for it) of the training set for a neural network?

What you are referring to is the topic of computational learning theory. One way to measure it is to use Vapnik-Chervonenkis dimension.

There are some results on VC dimension bounds of neural networks: Vapnik-Chervonenkis Dimension of Neural Nets

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.