Imagine a dataset containing billions of data. Some of them will never be used because of its length. That leads to my question: is it better for a Deep Learning model to train constantly new data of this huge dataset, or in opposite, take a reduced amount of data and create x epochs of them?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If possible, it's better to use the larger one, as it'll represent the data distribution better. It may not be computationally feasible to use all of it though. In that case, the large data can be sampled each time while forming the mini batches.
