# How to understand the predicted **negative** values by Kernel Regularized Least Squares (KRLS)?

I am learning the prediction algorithm, Kernel Regularized Least Squares (KRLS). The predicted values are listed in the follows:

$$\hat{y} = K((K + 1 \times I)^{-1}y)$$

For example, I have 100 samples, the independent matrix $X \in R^{m \times n}$, and the corresponding dependent variable $y \in R^{m \times 1}$, where $y \in \{0, 1\}$. I would like to build KRLS model, and predict the corresponding outputs. The predicted output values are real values. The larger values should tend to be label 1 and the smaller values should tend to be label 0, right?

From the $X$ matrix, I obtain the kernel matrix $K$, all components of $K$ are positive, and the dependent variable $y \in \{0, 1\}$.

I notice that some of the predicted values $\hat{y}$ are negative, how to interpret such negative results? These negative values are closer to label 0?

If the the predicted values are ranged $[0, 1]$, then I can say, the larger of the $\hat{y}$, it is closer to label 0, whiles the least of the $\hat{y}$, it should be closer to 0. I can consider the predicted values as the probability scores.

But, for the predicted negative values, I do not know how to interpret it.

I may do not understand the KRLS algorithm, any points are appreciated.

Thanks.

• You may need to provide some more details. A small reproducible example would be helpful. – Glen_b -Reinstate Monica May 14 '17 at 2:55
• @Glen_b Thanks. I upload my data into google drive here. The $K$ matrix is 445-by-445 and $y$ is 445-by-1. I got the predicted values by the formula: $\hat{y} = K(K + 1 \times I)^{-1}y$. Then the 100th predicted values is: -2.709858e-05. download K here and download y here – Kevin May 14 '17 at 12:33

Yes, you most likely should take those to be 0. Note that you are using a linear model (in kernel space) so you can definitely get values outside the $[0,1]$ range. Practically speaking, if you're interested in discrete output, you would likely be better off just using a classifier rather than regression - this was referred to by (I believe) Scholkopf et. all in their OC-SVM paper as not solving a more general problem than one has to (that was in the context of low density rejection as opposed to full density estimation).