5
$\begingroup$

I am trying to optimize the hyperparameters for a Gaussian process. I am using a squared exponential kernel, where I am optimizing three parameters.

$$k_y(x_p,x_q) = \sigma^2_f \exp\left(-\frac{1}{2l^2}(x_p-x_q)^2\right) + \sigma^2_n\delta_{pq}$$

As described by Rasmussen I am maximizing log marginal likelihood using its derivatives, but for some combination initial values of hyperparameters, I am occasionally getting negative values of $\sigma_f$ and $\sigma_n$. Are the negative values acceptable?

Many of the times people optimize the parameters in log domain to avoid negative values of hyperparameters, but I do not understand how ppl do that.

$\endgroup$
1
  • 3
    $\begingroup$ Clearly, based on the form of your kernel, a negative value of $\sigma_f$ and $\sigma_n$ is equivalent to their absolute value. $\endgroup$ Aug 23, 2017 at 13:34

1 Answer 1

4
$\begingroup$

It's "acceptable" in the sense that, if you just take the absolute value, you'll get the same model out, so just do that and then it doesn't matter.

But optimizing in the log domain often works better for these kinds of parameters anyway. You do that just by reparameterizing your kernel as $$ k_y(x_p, x_q) = \exp(s_f) \exp\left( - \frac{1}{2 l^2} (x_p - x_q)^2 \right) + \exp(s_n) \delta_{pq} $$ and then optimizing $s_f$, $s_n$ instead of $\sigma_f$, $\sigma_n$. (You might want to do the same for $l$.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.