2
$\begingroup$

As per the usual definition of epoch and iteration, it takes multiple iterations to complete 1 epoch (for example see https://stackoverflow.com/questions/4752626/epoch-vs-iteration-when-training-neural-networks).

So the relationship between epochs and iterations should be always #iterations >= #epochs. However in the h2o deep learning implementation, we observed that the number of epochs is higher than the number of iterations (as shown in the image).

Example image where epoch > iteration

This leaves the question: How are the epochs and iterations defined in a h2o deep learning perspective?

$\endgroup$
1
$\begingroup$

In H2O Deep Learning, the term "iteration" refers to a MapReduce iteration. There are two related parameters:

  • train_samples_per_iteration (usually best to leave the default)
  • score_each_iteration (False by default; set to True to get more frequent scoring)

The term "epoch" has the traditional meaning of one pass through all the training examples.

There is more information in the section 5.2.4 of the H2O Deep Learning Booklet.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Here is quite a useful source that answers your question. H2O defines an epoch as each time gradient descent is carried out (ie. weights and biases are changed).

The number of epochs used can be changed by the Epochs = argument. We can say that an epoch is carried out for each "batch", where a "batch" is a group containing training data. By default, a batch contains all training data, so each epoch uses all training data.

However, if you want only some of the data to be used each epoch, you have to change the size of a batch using the mini_batch_size = argument.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.