0
$\begingroup$

I am comparing multiple regression machine learning algorithms (MLA) for a project. I have been reading Geron's excellent book 'Hands-On Machine Learning with Scikit-Learn & Tensorflow'. He mentions that you should not spend too much time tweeking hyper-parameters first, just get a few good models together, then work on tweeking the hyper-parameters for each one.

So does this mean that I should find the best MLA for my data using the default parameters, discard the bad ones and only focus on tweeking the best model. Is there not a chance that the discarded regressors could out perform the chosen model if their hyper-parameters were tweeked? I am only asking as I am trying to work out the correct order in which KDD processes should be run.

Any advice welcome.

$\endgroup$
0
$\begingroup$

It doesn't mean that you should use them with default parameters (they're sometimes good by the way). You need to quickly try a few settings on many MLA and deep dive on best of them, he says. Actually, he's giving heuristics. Of course, you might be discarding the optimal model on your search path, but Geron, and I agree with him, is trying to say that having a coarse search of what is possible, it is easier for you to later focus on algorithms that most matter for your data. As the name suggests, heuristics don't tell the truth, but try to approach towards it and speed up the process.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.