I am trying to better understand “tuning the hyperparameters”. I understand how to use GridSearchCV, I found the below explanation useful:
“As we do not know whether those parameters affect each other, doing it right will require that we train a classifier for every possible combination of all parameter values. Obviously, this is too tedious for us.” (from ‘Building Machine Learning Systems with Python’).
Now, my question is model agnostic (let’s say we only have certain weights to learn from data). First we train the model (learn the weights), and then we use the GridSearchCV to tune the hyperparameters…
How do we know that the hyperparameters do not affect the parameters, or that the parameters do not affect the hyperparameters?
Let’s say I train a model to calculate the weights (with default hyperparammeters) . After than, I tune the hyperparemeters… If I re-run the model with the new tuned hyperparameters, am I assured 100% that I will get the same weights?