0
$\begingroup$

Suppose a bivariate process is observed $(X_1, Y_1), \ldots, (X_n, Y_n)$ that follows the probability model of ordinary linear regression

$$Y_i = \alpha + \beta X_i + \epsilon_i, $$

and assume $\beta> 0$.

We further are interested in predicting response at a certain exposure level $X$. The distance for an observed $X_i$ from the observation level is noted as $\delta_i = |X_i - X|$, and its order statistic is $\delta_{(i)}$ so that $\delta_{(1)}$ is the closest observed exposure level to the target prediction $X$ WLOG. The rank order set index for the $k$-nearest neighbors is $i: \delta_{(j)} \le \delta_{(k)}$

Take the prediction of the response for a certain level $X$ as

$$ \hat{Y} = \sum_{i: \delta_{(j)} \le \delta_{(k)}} Y_i / k$$

If $X \ge X_{(n)}$ then $\hat{Y} = \sum Y_i I(X \ge X_{(n-k+1)}) / k $, that is the $k$ responses that have $X$ top $k$ ranked response. But $E[Y_i I(X_i \ge X_{(n-k+1)})] < X \beta$. That is to say, all the $k$ neighbors are the highest rank orders of the exposure, and thus the prediction is consistently lower than necessary.

Is this a correct understanding of $k$-nearest neighbors? Is this a known or discussed problem? Are there proposed corrections?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.