This might come as a basic question. But I need to understand why do we need to tune the hyper parameters in a machine learning model instead of going into a different model altogether. Or to put it in this way, when do we decide we need to tune hyperparameters. What are the factors that are taken into consideration ?
$\begingroup$
$\endgroup$
1
asked May 9, 2020 at 6:44
-
$\begingroup$ In case you are interested, there is a nice lecture by Andrew Ng precisely on the topic of hyperparameters selection for NN : coursera.org/learn/deep-neural-network $\endgroup$– Camille GontierCommented May 9, 2020 at 8:26
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Why? To reach to the somewhat highest performance of a model, you need to try different hyperparameters
When? whenever you find an "appropriate" model for your task or made a architecture of a model (e.g. in artificial neural networks) then you need to tune hyperparameters to make sure that the model could make good enough predictions. It is when you could not improve the performance of the model by changing the model architecture.
There are different approaches for tuning of hyperparameters such as grid search and random search that you could choose based on you preferences. The point is that the "appropriate" model may differ from one application to another.
$\begingroup$
$\endgroup$
First of all, model with different hyperparameters would be a different model.
When you buy shoes, you need find appropriate brand, model, color, shape, and size. Same as there’s no one-size-fits-all shoes, applies to hyperparameters as well. There are algorithms that aren’t very sensitive to hyperparameters, or don’t have them, but for many algorithms you have no guarantees whatsoever that the set of parameters that worked for one dataset, would work for another.
Of course, this doesn’t mean that you always need to do proper tuning, using some sophisticated tuning algorithm. In many cases it is enough to “try different values”, or even make educated guesses before fitting the model to the data (e.g. for choosing depth of decision tree).
