0
$\begingroup$

I have to implement 3 Neural Networks, an MLP, an RNN and an LSTM, for a "Future Location Prediction" problem and it's the first time I am in touch with NNs.

I would like to ask whether there is a way to automatically find the best configuration for their parametres, aca number of layers, number of nodes in each layer, dropout, activation and loss functions, as well as optimiser.

If there is no such solution, would you suggest a better way than blind trying?

Thanks in advance!!

$\endgroup$

1 Answer 1

3
$\begingroup$

The two most common methods are Grid Search and Random Search. For both of these you specify a set or a range of hyperparameter values and then iteratively try combinations of hyperparameters to find the best combination. For a grid search, a set of values is provided for each hyperparameter to be tuned and each combination is tried systematically. For a random search, the values are selected randomly and tried, until a stopping condition is met. Typically, cross-validation is used to test the various hyperparameter combinations.

If you're using Python, there are several packages that can help you do this. If you search for "hyperparameter tuning python" you will find plenty of information on these packages and how to use them. To get started, have a look at Jason Brownlee's Machine Learning Mastery blog:

$\endgroup$
1
  • $\begingroup$ Do be aware though that searching over model architectures and any hyper-parameter that may be involved is a bit of a recipe for over-fitting the model selection criterion and the model that is selected by the procedure may be worse than the model you started with in extreme cases. For "shallow" neural networks, I would recommend using either pruning or growing methods rather than architecture selection, or just use a large network and regularisation (although that is not a panacea). $\endgroup$ May 10 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.