Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am aware that Bayesian models and K-nearest neighbor both suffer in prediction results, especially when the dataset size does not scale exponentially to the number of dimensions.
However, does the curse of dimensionality apply to linear regression classifiers as well?
Short answer is yes, according to my knowledge most linear regression models (even if used for classification) suffer from the "curse".
However, some regularised regression models are well suited for the case features >> samples.
See Lasso (and its grouped and sparse variants) and elastic net. In general, what they tend to do is dropping many features (ie, many coefficients will be 0) so that, in practice, they do both feature selection and regression.
Have a look, for example, at scikit-learn guides and implementations to see what I mean.
