1
$\begingroup$

Say I have some two dimensional data, for which I am trying to fit a Gaussian process. In scikit-learn, I can build an RBF kernel as follows

K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + sklearn.gaussian_process.kernels.RBF(length_scale=0.9)

or

K=sklearn.gaussian_process.kernels.RBF(length_scale=[0.1, 0.9])

Whats the difference between these two kernels, when should I use one over the other?

$\endgroup$
0
$\begingroup$

K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + sklearn.gaussian_process.kernels.RBF(length_scale=0.9)

Defines two kernels, and you will be basically using one kernel for each feature. Each kernel will basically act as an infinite degree polynomial for a dimension.

K=sklearn.gaussian_process.kernels.RBF(length_scale=[0.1, 0.9])

Defines one kernel that uses both features with each feature having its own length scale: 0.1 and 0.9 respectively. This kernel now not only acts as an infinite polynomial for each feature but all also includes all pairwise and higher order interactions between the features.

You want option two, that’s what people often use, for example in ML or spatial stat.

In case you wonder where those polynomials and interaction are calculated, that’s a feature of rbf kernel, there are many papers out there explaining this. Hint it’s because of exponent.

$\endgroup$
  • $\begingroup$ I have some data, where both kernels get perfect fit, but for some reason the log marginal likelihood prefers K=sklearn.gaussian_process.kernels.RBF(length_scale=0.1) + sklearn.gaussian_process.kernels.RBF(length_scale=0.9), do you know why this would happen? $\endgroup$ – Joel Nov 9 '19 at 1:38
  • $\begingroup$ Perfect fit does not sound good unless it’s a noiseless simulation or something. Putting that aside, as to why model prefers one to the other, there are many possibilities one that immediately comes in mind is that the amount of variance explained by features is very different and there is not much interaction between them contributing to the response, then model will clearly prefer the version you mentioned because it can tune a variance param per features rather one var param for both. $\endgroup$ – NULL Nov 9 '19 at 1:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.