How to obtain the class conditional probability when using KNN classifier? Using the KNN classifier, given a test data-point $x$, how do we get the probability of membership of $x$ to each class $y_i$, that is the probabilities $P(y_i | x)$ for $i = 1, 2, .., n$ (where $n$ is the number of classes).
 A: k-NN classifiers do not output probabilities.You would need to transform distance to a probability yourself, for example by fitting a logistic regression model on the distance.
The output of a k-NN classifier is in terms of distance of $\mathbf{x}$ to nearest member, e.g. $f(\mathbf{x}) = d \in \mathbb{R^+}$. You can transform this distance $d$ to a probability using logistic regression, e.g. find optimal coefficients $\beta_0$ and $\beta_1$ such that
$$g(d) = \frac{1}{1+exp(\beta_0+\beta_1d)} \in [0, 1],$$
is a well calibrated probability of class membership. Your total classification would then resemble the following $g(f(\mathbf{x}))$.
A: I'd like to share a different way to get the class conditional probability, which is different from what Marc proposed.
For an instance $\mathbf{x}$, assume we have $N_i$ instances belong to class $Y_i$ in the neighborhood. Then we define 
$$P(Y_i \, | \, \mathbf{x}) = \frac{N_i + s}{K + Cs},$$
where $K$ is total number of instances in the neighborhood, $C$ is the total number of classes, and $s$ is the smoothing parameter. The smoothing is used to avoid 0 probabilities. 
It is kind of a nonparametric way to get the conditional probability.
