2
$\begingroup$

I want to implement kernel ridge regression in R. My problem is that I can't figure out how to generate the kernel values and I do not know how to use them for the ridge regression.

Before going to the code can anybody help me with explaining how kernel ridge regression works conceptually?

  • What do I have to do step by step to be able to implement ridge regression?
  • Do I have to obtain kernel values for every independent variable in my training set, or for every data point?
  • And how can I use these values for ridge regression?

I want to use the following kernel function:

kernel.eval <- function(x1, x2, ker) {
  k=0
  if (ker$type=='RBF') {
        # RBF kernel
        k = exp(-sum((x1-x2)*(x1-x2)/(2*ker$param^2)))
      }
      else {
        # polynomial kernel
        k = (1+sum(x1*x2))^ker$param
  }
  return(k)
}

Furthermore, I know that the formula for ridge regression is:

myridge.fit <- function(X, y, lambda) {
    w = solve((t(X)%*%X) + (lambda*diag(dim(X)[2])), (t(X)%*%y))
    return(w)
  }

Example training data:

           [,1]       [,2]
[1,] -1.3981847 -1.3358413
[2,]  0.2698321  1.0661275
[3,]  0.3429286  0.8805642
[4,]  0.5210577  1.1228635
[5,]  1.5755659  0.2230754
[6,] -1.2167197 -0.6700215

Example testing data (I do not know if I need these at this moment):

      [,1]   [,2]
[1,] -2.05 -2.050
[2,] -2.05 -2.009
[3,] -2.05 -1.968
[4,] -2.05 -1.927
[5,] -2.05 -1.886
[6,] -2.05 -1.845

Is anyone able to help me with the first step(s). I have to do ridge regression for a RBF kernel as well as a polynomial kernel.

$\endgroup$
  • $\begingroup$ Are you asking for help with the code, or with how kernel functions work with ridge regression? $\endgroup$ – gung Nov 23 '15 at 3:27
  • $\begingroup$ Both would be really helpful. I have a basic understanding of how kernel functions work with ridge regression, but I do not have enough knowledge of it to be able to fix this code $\endgroup$ – De Cas Nov 23 '15 at 3:29
  • $\begingroup$ Asking for code help is typically off topic here. You may want to edit your question to emphasize the conceptual aspects. You may get code help along with your answer here, or you can take what you learn & ask a coding question on Stack Overflow. $\endgroup$ – gung Nov 23 '15 at 4:33
  • $\begingroup$ I edited the question. I think my problem right now is that I know the purpose of kernel ridge regression, but I don't know the idea behind it. Do I have to obtain kernel values for every independent variable in my training set? Or for every datapoint? And how can I use these values for ridge regression? $\endgroup$ – De Cas Nov 23 '15 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.