5
$\begingroup$

Amazon ML (Amazon Machine Learning) is offered by Amazon as a general purpose supervised classification service. They don't even bother to mention what algorithms are being used, they just use the generic term "ML models".

From their Key Concepts page:

An ML model is a mathematical model that generates predictions by finding patterns in your data. Amazon ML supports three types of ML models: binary classification, multiclass classification and regression.

Upon digging deeper, they state in their FAQ:

Amazon Machine Learning currently uses an industry-standard logistic regression algorithm to generate models.

Based on this, one would assume that logistic regression can pull off anything that an SVM, an RF or a Neural Net can. Is that the case?

Isn't logistic regression limited to linearly separable models? Isn't it just a special case of Neural Net (i.e. one layer perceptron) ?

And is it really an "industry standard", and is it really so versatile that Amazon would favor it over something more complex ?

$\endgroup$
  • 1
    $\begingroup$ Maybe they mean they do GLMs, which can do regression, and with the right family/link binary classification (logistic regression), and multiple logistic regressions can be used to do multi class regression. They don't claim that they can "pull off" anything a Neural Net can do. (SVM's, in my opinion, are overrated, and have no probabilistic interpretation, so no great loss there.) $\endgroup$ – Wayne Jan 6 '17 at 22:38
  • $\begingroup$ @Wayne it's true that they don't make the explicit claim. But their product description clearly indicates that is supposed to be a one-stop shop for all of your supervised classification needs. And if they're using GLM's, why not say so? "industry-standard logistic regression" are their words not mine. $\endgroup$ – Skander H. Jan 6 '17 at 22:45
  • $\begingroup$ My guess would be that many non-statisticians have heard of Logistic Regression, while few have heard of GLMs. Or maybe they wrote the Key Concepts and FAQ were written at different times by different folks and don't quite match up. $\endgroup$ – Wayne Jan 6 '17 at 22:49
  • $\begingroup$ For classification, Kernel SVMs have the advantage of the possibility of sparseness, and SVMs are also applicable in regression, with epsilon-insensitive losses. Also, ordinary SVMs are $\ell_2$-regularized. I guess a $\ell_2$-regularized Kernel Logistic Regression could be a good alternative. $\endgroup$ – Firebug Jan 7 '17 at 2:30
5
$\begingroup$

That AML uses logistic regression for all cases, if true, is somewhat surprising to me, since logistic regression is not generally the most powerful classification tool. Perhaps AML uses it because training the logistic regression model would require much less of Amazon's resources than, say, training a boosted decision tree -- though as a customer, processing power is what I'd think I was paying Amazon for.

If they are indeed using only logistic regression, they are possibly using augmented versions for increased performance. For instance, they could conceivably be training a GAM with the logistic loss function, to account for nonlinear relationships via smoothing splines, or modeling interactions through some automated procedure.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.