Can large # of epochs or smaller batchsize compensate for smaller data size in training lstms

I have about 40 time series (40 products) of weekly sales for 3 years ( = 156 data points for each series). So, in total I have about 6240 data points. To train a stateful or stateless lstm for predicting the sales (assuming I have yearly seasonality) and this smaller data might be insufficient.

Can I compensate the smaller data size by training for large # of epochs or having smaller data size, if I want to go with lstm?

• Compensate for what?
– Sycorax
Nov 6, 2020 at 2:19
• @Sycorax compensate for the smaller data size, as in general deep learning models needs lots of data (though amount of data is dependent on complexity and variability in data).
– tjt
Nov 6, 2020 at 2:21

You can't compensate in this way because having more epochs doesn't give you more data -- the sample size stays the same size.

In a certain sense, the opposite is true -- early stopping only trains a network for a certain number of iterations but stops before the parameters move enough to overfit. This can be shown to be equivalent to $$L^2$$ regularization.

If you've got a small amount of data, the best thing to do is collect more data.

If that's not possible, then your best bet is to use simpler models, e.g. regression strategies.

Regularization can also help avoid overfitting.

• Thank you so much, As with all your great answers on this site, this is useful, learnt a lot from all of them. I will wait if this get any more answers and will accept your answer.
– tjt
Nov 6, 2020 at 2:33
• Won't having more epochs and increased batch size result in more updates to the weights (because updates are made per batch) and there by resulting in achieving convergence (because of more updates ?). I understand that it may/will overfit, but will it at least approximate the function ?
– tjt
Nov 6, 2020 at 2:37
• Overfitting describes a scenario where the estimated function $\hat{f}$ does not approximate the desired function $f$ very well.
– Sycorax
Nov 6, 2020 at 2:38
• In general, in deep learning, how do we know if the size of training data is insufficient for the modeling task or how to know if adding more data will help approximate the function better and improve model performance (without actually adding more data, is there a way to check).
– tjt
Nov 6, 2020 at 3:47
• This seems like a perfectly fine question to ask on its own.
– Sycorax
Nov 6, 2020 at 4:35