In 1999, Beyer et al. asked, When is "Nearest Neighbor" meaningful?
Are there better ways of analyzing and visualizing the effect of distance flatness on NN search since 1999?
Does [a given] data set provide meaningful answers to the 1-NN problem? The 10-NN problem? The 100-NN problem?
How would you experts approach this question today?
Edits Monday 24 Jan:
How about "distance whiteout" as a shorter name for "distance flatness with increasing dimension" ?
An easy way to look at "distance whiteout" is to run 2-NN, and plot distances to the nearest neighbor and second-nearest neighbors. The plot below shows dist1 and dist2 for a range of nclusters and dimensions, by Monte Carlo. This example shows pretty good distance contrast for the scaled absolute difference |dist2 - dist1|. (The relative differences |dist2 / dist1| → 1 as dimension → ∞, so become useless.)
Whether absolute errors or relative errors should be used in a given context depends of course on the "real" noise present: difficult.
Suggestion: always run 2-NN; 2 neighbors are useful when they're close, and useful when not.