There is significant literature on using deep nets on classification problems. I'm currently trying to apply a deep learning to a regression problem, and getting mediocre results compared to tree based methods. I'm assuming that theoretically, a deep net should offer similar if not superior performance if the correct architecture, activation functions, etc. are specified. Is there any relevant literature, or practical advice available for deep learning as it applies to regression?
Edit Just to give some context, I'm talking about high-dimensional data with mixed types (categorical, continuous) and where the features may be obfuscated, and the goal is to build the most accurate model according to the RMSE on a holdout set. Think benchmarks, machine learning contests, etc.