I am making a system that outputs an estimate using some data input (think speech recognition). When testing my solution, I find that low values of knn work very well for data with low "noise" and badly for data of high "noise", however, high values of knn work very well for data with high "noise" but badly for data of low "noise", where high "noise" is akin to a crowded room and low "noise" is akin to one person speaking clearly.
Ideally, I would implement a system that would figure out an estimated best k value to use depending on the number of unique neighbours. I know that k is generally set to the square root of the number of data points, however, my results show that there is no one right value of k to use. I am simply assuming that this is something to do with the number of unique neighbours, however, I can't get my head round the reason as to why this is.