I'm trying to understand the symptom of sparsity when increasing the number of features. I followed this article
In this article, the writer claims (refer to Figure 10 please) that
These samples are difficult to classify because their feature values greatly differs (e.g. samples in opposite corners of the unit square).
Then, obviously when increasing the dimension (Figure 11), the ratio between the sphere and the feature space is increased.
The author says that
In high dimensional spaces, most of the training data resides in the corners of the hypercube defining the feature space. As mentioned before, instances in the corners of the feature space are much more difficult to classify than instances around the centroid of the hypersphere
So mainly I don't understand two things:
- Why is it harder to classify "cornered samples"? Aren't they more easy to classify because some features are more significance?
- Why is it true that "In high dimensional spaces, most of the training data resides in the corners of the hypercube"?