Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
All examples I found using deep belief or convolutional neural networks use them for image classification, chatacter detection or speech recognition.
Are deep neural networks also useful for classical regresion tasks, where the features are not structured (e.g., not arranged in a sequence or grid)? If yes, can you give an example?
The characteristics of images that makes them amenable to classification with a deep neural network is there are a ton of features (possibly millions if not billions of pixels with RGB, intensity, etc.) and if you have accurate labels, it's not noisy data. Cameras these days are very good and they aren't mis-measuring anything. Thanks to the Internet, we now have a lot of accurately labeled images. A deep network can express arbitrarily complicated functions, which is a problem with noisy data because you can very easily overfit the noise, hence why many learning methods tend to penalize complicated models. In the case of image recognition, however, the true function seems to actually be very complicated, we have no idea what the functional form looks like, and we don't even know what the relevant features are in many cases. A many-layer network can automatically discover and extract relevant features, which doesn't make it completely unique, but that is one attractive part of the model.
This doesn't mean you can't use deep networks to learn functions having nothing to do with images. You just need to be very careful about the downsides, mostly that it is very prone to overfitting, but also that it's computationally expensive and can take a long time to train (not as much an issue these days with parallelized SGD and GPUs). The other downside is you have very little to no model interpretability, which doesn't really matter for image classification. We're just trying to get computers to recognize the difference between a chimp and an orangutan. Human understanding of the formula doesn't matter. For other domains, especially medical diagnostics, policy research, etc., you want or might even need human understanding.
Sure you can use deep neural networks for many problems apart from image or speech recognition. The problem is if you really need it.
Deep neural networks are far more powerful than a simple MLP however they also take more resources and are more difficult to develop. Thus they are used in really complex domains. You could use them to solve easier problems but usually simpler models obtain good results too.
Using deep neural networks for easy problems will be like killing flies with a bazooka, sure you'll kill them but couldn't you find a simpler way?
I agree with davidivad's answer. But also I think the application of deep neural networks to images is that images (and, more importantly, labeled images) are relatively inexpensive to collect. In other domains, it can be very expensive to collect data on a large scale, especially within the constraints of a typical industrial or government enterprise. Compounding this issue is that in many applications, the phenomenon of interest is relatively rare, so there will be precious few examples to learn from, so even a relatively large-scale data collection effort might yield a small number of members of some class.
