Negative values of hyperparameters in Gaussian Process

Question

I am trying to optimize the hyperparameters for a Gaussian process. I am using a squared exponential kernel, where I am optimizing three parameters.

$$k_y(x_p,x_q) = \sigma^2_f \exp\left(-\frac{1}{2l^2}(x_p-x_q)^2\right) + \sigma^2_n\delta_{pq}$$

As described by Rasmussen I am maximizing log marginal likelihood using its derivatives, but for some combination initial values of hyperparameters, I am occasionally getting negative values of $\sigma_f$ and $\sigma_n$. Are the negative values acceptable?

Many of the times people optimize the parameters in log domain to avoid negative values of hyperparameters, but I do not understand how ppl do that.

Answer 1

It's "acceptable" in the sense that, if you just take the absolute value, you'll get the same model out, so just do that and then it doesn't matter.

But optimizing in the log domain often works better for these kinds of parameters anyway. You do that just by reparameterizing your kernel as $$ k_y(x_p, x_q) = \exp(s_f) \exp\left( - \frac{1}{2 l^2} (x_p - x_q)^2 \right) + \exp(s_n) \delta_{pq} $$ and then optimizing $s_f$, $s_n$ instead of $\sigma_f$, $\sigma_n$. (You might want to do the same for $l$.)