I am learning the prediction algorithm, Kernel Regularized Least Squares (KRLS). The predicted values are listed in the follows:

$$\hat{y} = K((K + 1 \times I)^{-1}y)$$

For example, I have 100 samples, the independent matrix $X \in R^{m \times n}$, and the corresponding dependent variable $y \in R^{m \times 1}$, where $y \in \{0, 1\}$. I would like to build KRLS model, and predict the corresponding outputs. The predicted output values are real values. The larger values should tend to be label 1 and the smaller values should tend to be label 0, right?

From the $X$ matrix, I obtain the kernel matrix $K$, all components of $K$ are positive, and the dependent variable $y \in \{0, 1\}$.

I notice that some of the predicted values $\hat{y}$ are negative, how to interpret such negative results? These negative values are closer to label 0?

If the the predicted values are ranged $[0, 1]$, then I can say, the larger of the $\hat{y}$, it is closer to label 0, whiles the least of the $\hat{y}$, it should be closer to 0. I can consider the predicted values as the probability scores.

But, for the predicted negative values, I do not know how to interpret it.

I may do not understand the KRLS algorithm, any points are appreciated.


  • 1
    $\begingroup$ You may need to provide some more details. A small reproducible example would be helpful. $\endgroup$ – Glen_b -Reinstate Monica May 14 '17 at 2:55
  • $\begingroup$ @Glen_b Thanks. I upload my data into google drive here. The $K$ matrix is 445-by-445 and $y$ is 445-by-1. I got the predicted values by the formula: $\hat{y} = K(K + 1 \times I)^{-1}y$. Then the 100th predicted values is: -2.709858e-05. download K here and download y here $\endgroup$ – Kevin May 14 '17 at 12:33

Yes, you most likely should take those to be 0. Note that you are using a linear model (in kernel space) so you can definitely get values outside the $[0,1]$ range. Practically speaking, if you're interested in discrete output, you would likely be better off just using a classifier rather than regression - this was referred to by (I believe) Scholkopf et. all in their OC-SVM paper as not solving a more general problem than one has to (that was in the context of low density rejection as opposed to full density estimation).

  • $\begingroup$ @motinisensor Thanks. These values are outside the [0, 1] range. Actually, I prefer the predicted real values instead of these discrete output, Since I would like to calculate the ranking-based metrics such as nDCG (Normalized Discounted Cumulative Gain), which need first sort the predicted values. One-class SVM is another option for me. Thanks. $\endgroup$ – Kevin May 14 '17 at 11:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.