In Gaussian Processes, we have kernel parameters. For example, for RBF kernel, we have $k(x, x')= \sigma_f^2 \exp(-\frac1{2l^2}(x-x')^2)$, where the $\sigma_f, l$ are the two parameters.
My questions are:
- Are they considered hyperparameters or parameters? I feel they are because they seem not attached to the model learning the data, but I cannot articulate it very well.
- If they are hyperparameters, then why do I see that their values are often estimated through maximizing the marginal likelihood. For example, in Kevin Murphy's Machine Learning book, or in the
fit()
function ofscikit-learn.GaussianProcessRegressor
here. Aren't hyperparameters not learnable and need to be found through grid search or random search?