5
$\begingroup$

I understand that defining the scope of what falls precisely under the scope 'machine learning' is impossible.

Yet, are there are insights from the machine learning literature that moves away from linear regression using OLS when the goal is predicting a continuous variable?

It seems that machine learning offers a lot more options (relative to models typically described in econometrics literature) when it comes to predicting categorical or binary outcomes, than continuous variables.

$\endgroup$
  • 6
    $\begingroup$ I feel that reading a review of machine learning methods could provide you with many examples. web.stanford.edu/~hastie/ElemStatLearn $\endgroup$ – Sycorax says Reinstate Monica Feb 18 at 17:37
  • 2
    $\begingroup$ No source, thus just a comment: ML (more or less) originated from categorical prediction. Many methods have been generalized to continuous predictions by now, but way back this was not the original intent. Under the hood, many methods still use categorical variables even if predicting continuous ones. $\endgroup$ – Eulenfuchswiesel Feb 19 at 10:38
23
$\begingroup$

It is not true. There's similar number of regression algorithms as classification algorithms in machine learning. Most of the classification algorithms have their regression counterparts: there's $k$-NN and $k$-NN regression, SVM's and SVR's for regression, random forest build of classification, or regression trees, XGBoost can be used for both tasks, there's an infinite number of regression neural networks, etc.

$\endgroup$
8
$\begingroup$

Are there are insights from the machine learning literature that moves away from linear regression using OLS when the goal is predicting a continuous variable?

There are many methods other than standard OLS for modelling continuous data. There are even linear regression methods other than OLS (TLS/Deming regression for one). Other classical methods include GLMs, GAMs, quantile regression. Regularised regression also modifies the OLS penalty and therefore is not "just" OLS.

I don't know why you think statistics had not moved past OLS without the help of machine learning. Perhaps it would be useful to read an introduction to statical learning.

$\endgroup$
0
$\begingroup$

IMO, deep learning is under the machine learning umbrella, in that it is deep machine learning, instead of "shallow" machine learning methods (e.g., OLS, KNN, SVM, Random Forest).

Deep learning and artificial neural networks can be used for regression problems, to add another OLS alternative path for you. Keep in mind that the more features/variables you have, the more data points you need for training and the more compute resources you will need to run the training on. Here are some good starting points:

1) https://machinelearningmastery.com/regression-tutorial-keras-deep-learning-library-python/

2) https://www.tensorflow.org/tutorials/keras/regression

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.