# Predicting response on boundaries of observations with KNN

Suppose a bivariate process is observed $$(X_1, Y_1), \ldots, (X_n, Y_n)$$ that follows the probability model of ordinary linear regression

$$Y_i = \alpha + \beta X_i + \epsilon_i,$$

and assume $$\beta> 0$$.

We further are interested in predicting response at a certain exposure level $$X$$. The distance for an observed $$X_i$$ from the observation level is noted as $$\delta_i = |X_i - X|$$, and its order statistic is $$\delta_{(i)}$$ so that $$\delta_{(1)}$$ is the closest observed exposure level to the target prediction $$X$$ WLOG. The rank order set index for the $$k$$-nearest neighbors is $$i: \delta_{(j)} \le \delta_{(k)}$$

Take the prediction of the response for a certain level $$X$$ as

$$\hat{Y} = \sum_{i: \delta_{(j)} \le \delta_{(k)}} Y_i / k$$

If $$X \ge X_{(n)}$$ then $$\hat{Y} = \sum Y_i I(X \ge X_{(n-k+1)}) / k$$, that is the $$k$$ responses that have $$X$$ top $$k$$ ranked response. But $$E[Y_i I(X_i \ge X_{(n-k+1)})] < X \beta$$. That is to say, all the $$k$$ neighbors are the highest rank orders of the exposure, and thus the prediction is consistently lower than necessary.

Is this a correct understanding of $$k$$-nearest neighbors? Is this a known or discussed problem? Are there proposed corrections?