Kernel design for Gaussian processes with multiple inputs

Question

How can I design a kernel function when there are multiple input variables and their degree of influence on the covariance with the target variable is different from each other?

For example, if the input vector is x, and the value of x is normalized, then using a kernel like rbf(x, x'), I think the percentage of influence of each input on the covariance will be equal. (Because it is calculated by the Euclidean distance between x and x') The definition is from sklearn:https://scikit-learn.org/stable/modules/generated/sklearn.gaussian_process.kernels.RBF.html

What kind of kernel design is commonly used when we don't want to make the assumption that the input variables have an equal impact? Should I add them up like rbf(x1, x1') + rbf(x2, x2')...?

Answer 1

What kind of kernel design is commonly used when we don't want to make the assumption that the input variables have an equal impact?

What you're probably looking for is a anisotropic kernel. The most typical manner in which this is implemented is via what gets called "automatic relevance determination" or "ARD". What this means is that each "input dimension" is permitted to have a different lengthscale parameter (for a problem with $d$ "input dimensions", the lengthscale parameter becomes a vector of length $d$).

In practice, this means the similarity between function values modelled by the kernel (which is essentially what the covariance kernel prescribes) is allowed to be more or less "sensitive" to different input dimensions.

Kernels accepting multidimensional inputs can be formulated with different structures (in less precise terms, the input dimensions can be "combined" in different ways; for example additively or multiplicatively). You may find it helpful to read this article and follow some of the references if you're interested in this. The canonical way is multiplicatively.

To get an anisotropic kernel in SKLearn, you should simply be able to pass the lengthscale as a vector of size $d$. See the "parameters" section here.