4
$\begingroup$

I am aware that Bayesian models and K-nearest neighbor both suffer in prediction results, especially when the dataset size does not scale exponentially to the number of dimensions.

However, does the curse of dimensionality apply to linear regression classifiers as well?

$\endgroup$
3
$\begingroup$

Short answer is yes, according to my knowledge most linear regression models (even if used for classification) suffer from the "curse".

However, some regularised regression models are well suited for the case features >> samples.

See Lasso (and its grouped and sparse variants) and elastic net. In general, what they tend to do is dropping many features (ie, many coefficients will be 0) so that, in practice, they do both feature selection and regression.

Have a look, for example, at scikit-learn guides and implementations to see what I mean.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.