3
$\begingroup$

I've been searching through numerous kernels used in Gaussian processes, and one common feature is that the covariance matrices always have only positive elements. Yet the only requirement on the covariance matrix itself is that it must be positive semi-definite (e.g. symmetric and all eigenvalues greater than 0). For the concept of negative correlation to even exist requires the existence of negative components in the covariance matrix. Yet none of the kernels I come across in the literature permit this possibility, and I am simply wondering if there is a theoretical reason for why kernels used in Gaussian processes do not have negative components?

$\endgroup$
2
$\begingroup$

Actually your linked source gives already an example where the kernel matrix can have negative entries, the linear kernel: $$ k(x,y) = a + (x - c)(y - c).$$ Other examples are given by dot product kernels such as $$ k(x,y)= <x,y>^n.$$

Your impression was probably formed because many kernels used in practice are radial basis functions $$ k(x,y) = g(\|x - y\|)$$ based on euclidean distance. For those all entries must be positive indeed, because $g$ has to be completely monotonic, which in particular requires $g$ to be positive. (See Theorem 7.13 of Wendland)

$\endgroup$
  • $\begingroup$ Before fully computing the matrix and checking for only non-negative eigenvalues, Is there a way to prove if a function is a valid kernel simply by analyzing its functional form (e.g. for the linear kernel you've written, can it be proven that covariance matrix produced by this function and evaluated at any point will always be positive semidefinite)? $\endgroup$ – Mathews24 Jul 17 '19 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.