1
$\begingroup$

I am quite new to kernel method, I am trying to estimate $y'$ corresponding to $x'$, given [x, y] data. I am using Gaussian Method for analysis with Kernel function: $k(x_1,x_2 ) =p_1\exp\{-p_2(x_1 -x_2 )^2\} $

Considering the x discritization: x = 0:50:1000; & $p_1 = 1;$ For $p_2 > 10^{-5}$ the end elements of kernel matrix start approaching zero and for $p_2 < 10^{-7}$ the diagonal elements of matrix starts becomes unity. For the specific range $10^{-7}< p_2<10^{-5}$, inverting the kernel matrix outputs warning in Matlab Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 5.089376e-18. i.e. Matrix has high condition number.

Now my doubts:

  1. How do I guess $p_1$?
  2. Is restiction $p_2$ between [10^{-7}, 10^{-5}] a correct approach? If yes how to counter the Matrix scaling problem?
x = 0:50:1000;
p1 = 1;
p2 = 10^(-7):10^(-7):10^(-5); 
for i = 1:length(p2)
    par = [p1 p2(i)];
    Cxx = GPcov(x, x, par);
    inv_Cxx = inv(Cxx);
end

.

function C = GPcov(s, t,ppar)
x1 = s(:);
x2 = t(:);
[n, D] = size(x1);
[m, d] = size(x2);
dist = zeros(n, m);

% --Calculating Euclidean distance ---------

x1Matrix = repmat(x1, 1, m);
x2Matrix = repmat(x2', n, 1);
dist = (x1Matrix - x2Matrix).^2;
C = ppar(1)*exp(-ppar(2)*dist);
end

Matrix for $p_1 = 1, p_2 = 10^{-9}$ p1 = 1, p2 = 10^(-9)

Matrix for $p_1 = 1, p_2 = 10^{-4}$ p1 = 1, p2 = 10^(-4)

If I am not clear enough kindly let me know.

Thanks

$\endgroup$
  • $\begingroup$ common solutions include placing constraints on $p_i$ and adding some small values to the diagonals to improve the numerical properties of the matrix. $\endgroup$ – Sycorax says Reinstate Monica Feb 1 '16 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.