5
$\begingroup$

Amazon ML (Amazon Machine Learning) is offered by Amazon as a general purpose supervised classification service. They don't even bother to mention what algorithms are being used, they just use the generic term "ML models".

From their Key Concepts page:

An ML model is a mathematical model that generates predictions by finding patterns in your data. Amazon ML supports three types of ML models: binary classification, multiclass classification and regression.

Upon digging deeper, they state in their FAQ:

Amazon Machine Learning currently uses an industry-standard logistic regression algorithm to generate models.

Based on this, one would assume that logistic regression can pull off anything that an SVM, an RF or a Neural Net can. Is that the case?

Isn't logistic regression limited to linearly separable models? Isn't it just a special case of Neural Net (i.e. one layer perceptron) ?

And is it really an "industry standard", and is it really so versatile that Amazon would favor it over something more complex ?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Maybe they mean they do GLMs, which can do regression, and with the right family/link binary classification (logistic regression), and multiple logistic regressions can be used to do multi class regression. They don't claim that they can "pull off" anything a Neural Net can do. (SVM's, in my opinion, are overrated, and have no probabilistic interpretation, so no great loss there.) $\endgroup$
    – Wayne
    Commented Jan 6, 2017 at 22:38
  • $\begingroup$ @Wayne it's true that they don't make the explicit claim. But their product description clearly indicates that is supposed to be a one-stop shop for all of your supervised classification needs. And if they're using GLM's, why not say so? "industry-standard logistic regression" are their words not mine. $\endgroup$
    – Skander H.
    Commented Jan 6, 2017 at 22:45
  • $\begingroup$ My guess would be that many non-statisticians have heard of Logistic Regression, while few have heard of GLMs. Or maybe they wrote the Key Concepts and FAQ were written at different times by different folks and don't quite match up. $\endgroup$
    – Wayne
    Commented Jan 6, 2017 at 22:49
  • $\begingroup$ For classification, Kernel SVMs have the advantage of the possibility of sparseness, and SVMs are also applicable in regression, with epsilon-insensitive losses. Also, ordinary SVMs are $\ell_2$-regularized. I guess a $\ell_2$-regularized Kernel Logistic Regression could be a good alternative. $\endgroup$
    – Firebug
    Commented Jan 7, 2017 at 2:30

1 Answer 1

Reset to default
5
$\begingroup$

That AML uses logistic regression for all cases, if true, is somewhat surprising to me, since logistic regression is not generally the most powerful classification tool. Perhaps AML uses it because training the logistic regression model would require much less of Amazon's resources than, say, training a boosted decision tree -- though as a customer, processing power is what I'd think I was paying Amazon for.

If they are indeed using only logistic regression, they are possibly using augmented versions for increased performance. For instance, they could conceivably be training a GAM with the logistic loss function, to account for nonlinear relationships via smoothing splines, or modeling interactions through some automated procedure.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.