I'm running regression using GPML with a covariance function being a sum of a Gaussian noise and a Squared Exponent (SE). Input is in R4 and both the input and the output are normalized.

I run optimization of hyper-parameters by minimizing the negative log marginal likelihood.

covSEiso is k(x_i,x_j)=sf^2*exp-(x_i-x_j)^2/(2*ell)

After optimization, ell is reasonable (1.28), but sf^2 is extremely small (5.6445e-005). The noise variance is also reasonable (0.3715).

Therefore, the covariance matrix is practically 0 and thus the regression result is just the prior mean.

What in the data or elsewhere could be causing the scaler (sf) to become so small?

Note that it's the WHOLE covariance matrix that is almost zero, not JUST the covariance entries which would indicate lack of "relationship" between different entries; the variance is also practically zeroed out.




Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.