1
$\begingroup$

I know that SVM and SVR have different purposes in that the former is used for classification while the latter is used for regression, despite the similarity in the concepts between the two. However, in classification, SVM appears to be popular while for regression, I am under the impression that SVR isn't that common. I say this because it doesn't seem to be the first choice when performing regression -- others would, say, employ Gaussian process regression first. The coverage of SVR in text books on machine learning isn't as in-depth as SVM, for example, and is usually just mentioned as an aside. Is there a reason why?

I am not 100% familiar with SVR as I am only working on classification problems and hence have only been using SVM.

$\endgroup$
0
$\begingroup$

SVR has an edge over traditional linear regression in that it can handle non-i.i.d. information sets and can model non-linear relationships. Its drawbacks (not necessarily in relation to SVM) though are that (1) its theory and the results it materializes are not so easy to understand and interpret, (2) it can be costly to implement because its parameters require technical know-how for proper calibration, and (3) there's nothing really to fall back on in order to estimate the uncertainty contained in the results SVR provides.

Despite these drawbacks and the impression you seem to have, SVR has been one of the most popular machine learning tools for a decade now, especially for the prediction of financial time series, which Gaussian processes can't handle (see Tay and Cao 2001, Cao and Tay 2003, Kim 2003). The reason why SVR hasn't replaced least squares regression in undergraduate programs is because linear regression models are easier to understand and sufficient for a wide variety of problems and datasets found outside of finance that are more Gaussian than what we are faced with in our field. Also least squares is unbiased of all estimators according to the Gauss-Markov theorem, whereas many machine learning models like SVMs tend to either be biased (underfit) or have high variance (overfit), which brings in drawback #2 listed earlier.

SVM is still more popular than SVR because, like you said, the purpose of SVM (classification) is completely separate and stand-alone from the purpose of SVR (real-value estimation). One is a spoon, the other is a fork. The biggest companies today are entirely built on the problem of classification (Google, Facebook: image, ads, word recognition), making classification a more abundant problem today than non-linear regression is.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.