0
$\begingroup$

I'm reading up on Convolutions in neural nets, and they seem like a neat and efficient way of finding "features" in the input. But am I right in thinking that a high enough number of layers and neurons layers in a plain old multi-layered dense net would "find" the same features? Or have I missed something?

Are there any metrics or estimates available on the difference in training/prediction time and cpu/memory requirements between a network with a CNN layer and a plain DNN for the same accuracy?

Thanks in advance!

$\endgroup$
3
$\begingroup$

What convolutional layers do, is they slide a kernel over the image as illustrated in this cheetsheet by Stanford CS 230:

enter image description here

It is not only more efficient (less weights, since they are shared), but also it seeks for same features in different regions of the picture. If you used dense layer in here, the model would need to re-learn discovering those features independently for each region of the picture. This makes it much more complicated, but also it needs you to have data where in training set the features are distributed in all regions of the pictures. So if you had thousands of cars on pictures in your training set, but if there were none in upper left corner of the pictures, your classifier knows nothing about cars if they appear in upper left corner. With CNN's it wouldn't matter. For more details, check also this tutorial from another Stanford course.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Aha! The example I was following used the MNIST Fashion set, where each item is bang centre of the image, so I hadn't thought about finding the item in different places. Thanks for sorting me out! $\endgroup$ – Matt Mar 26 '19 at 10:55
  • $\begingroup$ @Matt for Fashion MNIST DNN is probably enough, since it is a clean & simplified dataset, but it would make a difference with real photos. $\endgroup$ – Tim Mar 26 '19 at 11:21
2
$\begingroup$

In addition to what @Tim says (+1), DNN can exactly imitate CNN, because CNN is a constrained version of some DNN. You're sliding a filter throughout the image for example; but since you're sliding the same filter, you constrain that the weights multiplied with some pixel set will be the same with weights multiplied with another pixel set somewhere different in the image. Also, there'll be zero weights throughout the layer because an output neuron in the hidden layer is affected by only a set of input neurons. So, while your DNN needs to figure out how to do all these (with more parameters), CNN has built-in capability, which explains the efficiency compared to DNNs.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.