I am looking at this paper Designing Neural Network Architectures using Reinforcement Learning.

The paper discussed how to find the best network using Reinforcement Learning.

According to APPENDIX A(Page 12):
M refers to number of models. That means, according to action plan (as shown Table 1, Page 5), the number of models to train is fixed.

It may be 1500 number of models or 2000 number of models according to image size.

I am confused for the advantage of using Reinforcement Learning in this condition. I can train all 1500 models and can select manually the best performance model.

Why we need Q Learning strategy and complicated stuffs?


The search space of all possible model architectures is much larger than 2000 models -- there may be trillions of possible architectures, yet since we only have limited time, we can't afford to train all of these models.

If we can only afford to train a few thousand architectures, then how shall we intelligently select those models? Hopefully, with RL, we can do better than random selection.

Related papers in the same vein of this one: NAS and ENAS

| cite | improve this answer | |
  • $\begingroup$ Thanks Sir, for the discussion. So how you select this M=1500 trained model (number of states) when epsilon is 1 (in exploration). How we make sure the best network is in this 1500? Since there are trillions of possible architectures. $\endgroup$ – batuman Aug 22 '18 at 1:53
  • $\begingroup$ @batuman you can never guarantee that you find the best network. In fact it is virtually certain you will not be able to find the best network. $\epsilon = 1$ corresponds to completely random exploration and as $\epsilon \rightarrow 0$, you converge on what the Q-network has determined to be "optimal". $\endgroup$ – shimao Aug 22 '18 at 3:50
  • $\begingroup$ thanks Sir. So we rely on Q network and what Q network gives us is considered as best of those M trained network. May not be the best from those trillion numbers. Then one thing I am confused is that if we can predefine M (say 1500) is the total number of trained networks, we just run all 1500 and select the best one manually. Why do we need this Reinforcement Q Learning? What is the advantage? $\endgroup$ – batuman Aug 22 '18 at 4:03
  • $\begingroup$ @batuman where do you get the 1500 architectures from? If you just sample them randomly, only a small number of them may be any good. We use RL in order to sample good architectures. $\endgroup$ – shimao Aug 22 '18 at 9:02
  • $\begingroup$ yes I understood now. You sampled randomly when epsilon is 1. Then from the models randomly selected just look for those with good accuracy. I got 1500 from the paper. The number of trained model is 1500 when epsilon is 1. $\endgroup$ – batuman Aug 22 '18 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.