2
$\begingroup$

I have a regression problem to be solved using one of neural networks models, but I have a small dataset which contains 30 samples.

Which training mode is more suitable for such dataset: stochastic or batch?

Of course I need to split data into training/test sets, I think that 2 or 3 folds cross-validation will not work well in this case, Am I right?

$\endgroup$
3
  • $\begingroup$ What do you mean? You mean training linear regression? Just use ordinary least squares. $\endgroup$
    – Tim
    Jan 28 at 11:29
  • $\begingroup$ I mean, when using neural networks, what should I select for the training process: batch or stochastic? $\endgroup$
    – jojo
    Jan 28 at 12:38
  • $\begingroup$ Please read Tim's answer. This is far too little data for neural network (or any complex ML algorithm). $\endgroup$ Jan 28 at 15:32

1 Answer 1

4
$\begingroup$

If you have 30 samples, you have insufficient data to use neural network. You should choose much simpler model. In fact, even with traditional machine learning your options would be pretty limited. For example, if you look at the flowchart from scikit-learn's site, with less than fifty samples it recommends to gather more data.

"Choosing the right estimator" flowchart, first step is checking if sample size is >50, if no it goes to "get more data".

This flowchart is of course an oversimplification and probably a bit pessimistic. With thirty samples you could try using linear regression with one or two features, though keep in mind that this is a small sample size even for linear regression.

If you insist on using neural network, it is very likely to overfit and not guaranteed to give reasonable results, no matter what you choose. With that small sample you can try both approaches and choose the best one using cross-validation. But again, keep in mind that you would be validating on just few samples, so you can easily overfit to test set and end up with a useless model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.