Can k-NN be ensembled? Decision trees suffer from high variance. Random Forests were suggested as a way of ensembling DT to solve this problem, and were shown to perform well for several problems. 
k-NN are also high bias classifiers, so in theory the same approach could be used with them. But I haven't heard much about ensembles of k-NN learners. 
Is there a reason for this? can k-NN be ensembled and if not why? 
 A: Sure, k-NN can be ensembled. You could, for example, use resampling to generate different models (like with a Random Forest), or you could vary N, or you could use different functions for computing the distance. But, my experience is that k-NN rarely does well in high dimensional problems, so it would just be an ensemble of bad models, which isn't going to do well relative to an ensemble of good models.
A: I see four abstract ways to do so from simplest to more complex.

*

*By applying $k$-NN in different Random Projections latent space or other (eg Neural network autoencoder latent space) and combine them.
that is: $Ensemble(kNN_{raw},kNN_{projected}) $


*apply different colaborative filtering scores eg. average distance, mean distnace, max, or any other linear combination eg: $ score = w_1+d_{k_1} + w_2+d_{k_2} ...$ (for unseuprvised learning only), ann similarly combine them.


*use different bins of neighbors eg the bin_1: 1st-10th k neighbors, bin_2: 10th-20th.
and combine the score from bin_number $k$-NN.


*different distance definition (minkowski, manhatan etc)
Hope it helps
