Why doesn't deep learning work as well in regression as in classification? there is a lot of research where deep learning works so well with classification but not in regression field
SVR, tree-based approach is still good and I couldn't find good architecture about regression
well there is some scheme you have to follow when implementing deep regression but I want to know why it doesn't work well as classification
any good theory or explanation why deep learning (mlp) doesn't work as well as classification?
I want to dig into this problem
 A: In my opinion, the following might be one -but not the only- reason for the relatively low popularity of Deep Learning in regression problems:
Much of the success of modern Neural Networks comes from their ability to exploit the compositional nature of the world. This is, in perception problems such as image or audio analysis, features have an order (spatial or temporal) and local patterns aggregate to form higher level concepts and objects (e.g, a picture of a car is made of wheels and other parts, which are made of lower level visual features, which are made of basic shapes like edges, circles and lines, etc.). Modern Neural Networks such as Convolutional Neural Networks take advantage of this by learning increasingly abstract features in the deeper layers. These ideas are explained in more detail in these slides by Hinton et al. [1]
Conversely, classical regression problems consist of a number of non-ordered features, and the target value can be predicted fairly well with a shallow linear/nonlinear model of the input features. In some sense, this compositional property present in problems such as image classification or speech recognition is not present in problems such as "Predict the income of an individual based on their sex, age, nationality, academic degree, family size...".
This might explain why some of the regression problems where Deep Learning is more popular are those based on images (e.g., Age prediction based on face photography).
Of course, these are only some intuitive ideas, and a more formal analysis of this problem is certainly a nice research topic. Deep Learning is certainly a field where more theoretical guarantees and insights are needed.
[1] https://www.iro.umontreal.ca/~bengioy/talks/DL-Tutorial-NIPS2015.pdf
A: You can "use" deep learning for regression. You have to consider the following:


*

*You can use a fully connected neural network for regression, just don't use any activation unit in the end (i.e. take out the RELU, sigmoid) and just let the input parameter flow-out (y=x). Consider that a NN with one neuron without activation unit is basically a simple linear regression.

*More generally this is not what you want, you will be using a very complicated structure of chained linear regressions that can tend to overfitting. Ridge regressions, Lasso, SVR and Bayesian predictors would do a better job at it and you would have better control about what you want to do.

