14
$\begingroup$

All examples I found using deep belief or convolutional neural networks use them for image classification, chatacter detection or speech recognition.

Are deep neural networks also useful for classical regresion tasks, where the features are not structured (e.g., not arranged in a sequence or grid)? If yes, can you give an example?

$\endgroup$
  • 3
    $\begingroup$ Your first sentence brings up convolutional neural networks. It appears that you're confusing them with deep belief networks. They're not the same, although both are forms of neural networks. $\endgroup$ – MSalters Nov 18 '15 at 15:39
  • 1
    $\begingroup$ I would agree with @msalters, but would say that deep belief networks are truly deep networks, and have had limited success, whereas convolutional nets are more like a hybrid - adaptive image filters s in convolutional layers followed by shallow nn. $\endgroup$ – seanv507 Nov 18 '15 at 20:40
  • $\begingroup$ What do you mean by "observations" being "not structured (not arranged in a sequence or grid)"? Do you refer to images being "structured" in the sense that individual pixels are arranged on a grid? But then it's features that are "structured", not "observations" (those would be individual images)? $\endgroup$ – amoeba says Reinstate Monica Nov 18 '15 at 21:15
  • $\begingroup$ I would say all Convolution networks are deep, not all are deep networks are convolution, and similarly all deep belief networks are deep, not all deep networks are deep belief networks. Indeed you can have deep networks that are neither deep nor convolutional, they just tend to be hard to train. There is certainly scope for pointless debate on terminology though. $\endgroup$ – Lyndon White Nov 19 '15 at 1:34
  • $\begingroup$ You can't apply a convolutional network to nonstructured (not in sequence/grid etc) data. It basically doesn't make sense. A convolutional network is closely related to taking the Fourier transform of you input -- eg for sequences converting it from the time domain to the frequency domain. $\endgroup$ – Lyndon White Nov 19 '15 at 1:37
8
$\begingroup$

The characteristics of images that makes them amenable to classification with a deep neural network is there are a ton of features (possibly millions if not billions of pixels with RGB, intensity, etc.) and if you have accurate labels, it's not noisy data. Cameras these days are very good and they aren't mis-measuring anything. Thanks to the Internet, we now have a lot of accurately labeled images. A deep network can express arbitrarily complicated functions, which is a problem with noisy data because you can very easily overfit the noise, hence why many learning methods tend to penalize complicated models. In the case of image recognition, however, the true function seems to actually be very complicated, we have no idea what the functional form looks like, and we don't even know what the relevant features are in many cases. A many-layer network can automatically discover and extract relevant features, which doesn't make it completely unique, but that is one attractive part of the model.

This doesn't mean you can't use deep networks to learn functions having nothing to do with images. You just need to be very careful about the downsides, mostly that it is very prone to overfitting, but also that it's computationally expensive and can take a long time to train (not as much an issue these days with parallelized SGD and GPUs). The other downside is you have very little to no model interpretability, which doesn't really matter for image classification. We're just trying to get computers to recognize the difference between a chimp and an orangutan. Human understanding of the formula doesn't matter. For other domains, especially medical diagnostics, policy research, etc., you want or might even need human understanding.

$\endgroup$
5
$\begingroup$

Sure you can use deep neural networks for many problems apart from image or speech recognition. The problem is if you really need it.

Deep neural networks are far more powerful than a simple MLP however they also take more resources and are more difficult to develop. Thus they are used in really complex domains. You could use them to solve easier problems but usually simpler models obtain good results too.

Using deep neural networks for easy problems will be like killing flies with a bazooka, sure you'll kill them but couldn't you find a simpler way?

$\endgroup$
  • 2
    $\begingroup$ This is a non answer. What is easy what is hard? Predicting the stock market/extrapolating from limited examples/... THERE are lots of hard problems are deep nns good at all of them? $\endgroup$ – seanv507 Nov 18 '15 at 19:57
  • $\begingroup$ I haven't said that deep neural networks can solve anything. What I meant is that they are used in complex domains where you have a huge number of entries. I know that they cannot solve every problem but it is not the point in this question. The point is enphasizing that they could be applied to other problems apart from image/speech recognition but they have downsides worth considering in the cases where other models could be applied. $\endgroup$ – davidivad Nov 19 '15 at 22:01
5
$\begingroup$

I agree with davidivad's answer. But also I think the application of deep neural networks to images is that images (and, more importantly, labeled images) are relatively inexpensive to collect. In other domains, it can be very expensive to collect data on a large scale, especially within the constraints of a typical industrial or government enterprise. Compounding this issue is that in many applications, the phenomenon of interest is relatively rare, so there will be precious few examples to learn from, so even a relatively large-scale data collection effort might yield a small number of members of some class.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.