1
$\begingroup$

$\mathcal{K}\Big( \; (x,y), (x',y') \; \Big) = \sigma_f^2 \exp{ \frac{(x-x')^2}{2l^2 \cdot (y+y')^2} } $, where $l > 0$

The variance associated with each training point (given by a vector) is a function of a co-efficient along one of the dimensions. This seems to point towards heteroscedasticity. However, the above kernel (if valid) looks like it would cover the property.

So, is the kernel valid? If so, does that mean I don't need to use a heteroscedastic GP framework to model variance as a function of the input?

$\endgroup$
2
$\begingroup$

With some help from a friend, I performed a quick test on MATLAB to check this. To find whether or not this was a valid kernel, I needed to check whether the resulting covariance matrix was positive semi-definite (PSD). I used the following test.

clear;    
num_points  = 100;

xrange = [-100, 100];
% Sample from a uniform distribution in the given range
x = xrange(1) + 2 * xrange(2) * rand(num_points, 1);

yrange = [1, 20];
% Sample from a uniform distribution in the given range
y = yrange(1) + 2 * yrange(2) * rand(num_points, 1);

cov = zeros(num_points);
for i = 1:num_points
    for j = 1:num_points
        num = (x(i)-x(j))^2;
        den = ( ( y(i) + y(j) ) );

        cov(i, j) = exp(- num / den);
    end
end

eval = eig(cov);
min(eval)

In my case, since the smallest eigen value turned out negative, the matrix is not PSD.

Note: This is a "rejection by sampling" test, and by itself cannot serve as conclusive proof that the covariance matrix is PSD, but in my case, since the result was negative, this was sufficient.

$\endgroup$
1
  • 2
    $\begingroup$ You have to careful with numerical issues here. I’ve seen similar tests “show” that the usual Gaussian kernel is not PSD. You are taking a bunch of exponentials and computing an Eigen decomposition of a large matrix, there is a lot that can go wrong. $\endgroup$ – guy Feb 26 '19 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.