I am trying to train a deep autoencoder (applies also for other architectures) in the following way:
Step 1) I start with a fix dataset of e.g. 10k samples.
Step 2) A training "loop" consists of 1000 epochs going through the whole dataset with mini-batches of size e.g. 64.
Step 3) After one "loop" the trained AE is used to generate more data, which (after some processing) is added to the dataset.
After, steps 2-3 are repeated for a predefined number of loops.
Question: What would be the best way to choose the number of iterations for one epoch?
a) Calculate it as:
num_iter = len(dataset)/batch_size, which increases everytime new data is added.
b) Keep it constant e.g.
num_iter = 200