I have developped a DL model to classify images and I am trying to optimize the hyperparameters. Reading the literature, I have found the ASHA algorithm and its implementation in SHERPA, like reported here. In the example there is something that I cannot understand. I have a maximum resource $R=9$ and a reduction factor $\eta = 3$, a minimal resource $r=1$ and an early-stopping rate $s=0$. To have a finished configuration, as reported in the link, I need to train 9 configurations for 1 epoch, then I select $3$ of them and train for $3$ other epochs, and finally training the surviving configuration for additional 9 epochs. Therefore, the number of trials should be $9+3+1 = 13$, but in the link are $12$. I do not understand.
Add a comment
|