I'd like to use Bayesian Optimization to tune the hyper parameters of a feed-forward neural network.
Among these hyper parameters, there is the number of hidden layers in the network, as well as the number of nodes in each layer. The issue is that the number of hyper parameters depends on the value chosen for the number of layers. For example, with a single hidden layer, there is only one parameter for the number of nodes. With five hidden layers, there are five number of nodes to choose.
Is there are smart way to handle this? Or do I have do choose between optimizing the number of layers with a fixed layer size and optimizing the number of nodes in each layers with a fixed number of layers?