I was trying to understand the difference between statistical regression VS machine learning regression. My background is from Economics and learned regression from statistical point of view for the first time. I learned machine learning later on and it also had regression. There might not be clear distinction between two but wanted to ask major differences.
What I can say (I might be wrong) now is there're from different areas and the model is different where statistical regression represents outcome consists of a set of independent variables with an error term whereas machine learning regression consists of outputs and inputs. Also, ML regression requires the process of training, testing to predict the unseen data sets but statistical regression doesn't require this process but analyze the past data and derives out the parameters (similarly weights in ML).
An intuitive explanations (no math or complex equations) using example might be helpful!
-
$\begingroup$ same thing except ML folks dont bother about assumptions. i doubt they even know about them $\endgroup$– AksakalCommented Mar 16, 2021 at 22:42
-
2$\begingroup$ Much of why we care about assumptions in statistics has to do with inference, not prediction. $\endgroup$– DaveCommented Mar 16, 2021 at 23:08
1 Answer
There really isn't much of a difference. A strained distinction between the two might be consideration of the data generating process (what statisticians call the likelihood). Statisticians care about this because different likelihoods lead to different types of inference. A hotly debated example of this would be the linear probability model (essentially linear regression on binary data) versus logistic regression. I won't spark the debate here, only offer it to you for future research. In general, machine learning doesn't seem to make many statistical assumptions about the data.
Leo Brieman's Two Cultures paper does a better job of elaborating on this difference. To the extent there is a difference between the two (I'm not certain there really is), this would be it.
