# Is there a simple method to calculate the minimum number of datalines needed for a machine learning algorithm with x variables?

Here is the problem: I am making a machine learning algorithm that takes the inputs and outputs of some software I've written, and I don't know how many datalines to produce to get results that are a 'good' fit. I realize the answer is 'the more the better' but I'm looking for any sort of minimum requirement. I also realize that the greater the number of variables the greater the number of datalines required.

So I'm looking for a rule of thumb for number of variables to number of datalines.

While there is no hard minimum requirement for the training data you need,

• There is a rule of thumb that says that with 5* cases per variate in each class you can usually have a stable linear classifier. Now, stable doesn't guarantee good, but unstable will always be bad. Note however, that in many disciplines with very wide data, good models are trained with far less samples.
* I've also seen 3 and 6, but that's the same order of magnitude.
• you can calculate minimum sample sizes required for testing: usually, building a good model is not sufficient. You also need to show that the model performs well. For relatively easy problems for which, however, only small sample sizes are available, this can actually be the harder part of the problem.
Not sure whether it applies to your scenario, though: you may be able to generate enough independent rows that this is not the concern.
Note also that classification performance measures that are proportions of test cases (overall accuracy, sensitivity, specificity, predictive values, precision, recall, etc.) are subject to very high variance and thus need large test sample sizes. This can also cause considerable difficulties with the learning curve (see the paper below for illustrations). Frank Harrell will tell you that better measures, e.g. Brier's score, should be used (see e.g. the discussion linked below). However, depending on your field, you may have to report the proportions.

One purpose (among many) of training is to estimate the out-of-sample error Eout given the obtained in-sample error Ein. There is such generalization in VC analysis: Eout <= Ein + Omega (also known as Hoeding inequality written in a probability format). So people tend to experimentally observe the changes of Eout and Ein. What they found is: as the sample number increases, the Eout would finally converged to Ein with a simple model; while Eout also converges from a huge value to a relatively constant value as well with a complex model, but still larger enough than Ein. Such VC analysis curve can be equivalently viewed as a bias-variance trade-off curve which indicates the variance decreases as the sample number increases. You can refer this slide for more details. On the other hand, the target complexity also influences on the generalization error. This slide shows a overfit measure as the number of data points changes. You can observe that if the target is complicated, increasing the sample number may not necessarily reduce the overfitting any more (the red region in the figures), yet it is still effective in reducing the overfitting if the target is not complicated.