2
$\begingroup$

This question basically focus on working of ANN and DNN. I really want to know, as both ANN and DNN may have multiple layer and also increase the number of hidden neuron. so, why DNN works better than ANN?

$\endgroup$
1
  • $\begingroup$ DNN work better than ANN for some types of task (e.g. image recognition), but for other tasks they are often no better (or perhaps worse) than ordinary ANNs (e.g. a lot of the UCI repository benchmark datasets). $\endgroup$ Dec 11, 2019 at 8:46

1 Answer 1

1
$\begingroup$

There is no real difference on the level of how to think about what 'neurons' are and how to hook them up, and the basics of the training rules used.

What has changed is that the field has advanced. There are now:

  • Better strategies for avoiding overfitting, e.g. dropout.

  • Non-gradient-saturating activation functions, e.g. ReLU.

  • New commonly implemented layers, e.g. convolutional, LSTM.

  • New computational frameworks, e.g. TF, pytorch.

  • New optimization strategies, e.g. Adam.

Papers that talk about DNNs will be newer than papers that talk about ANNs. They will use some or all of these developments. The combination of all the advances of the last 20 years (which builds on the previous work) does mean a step change in how good the results can be.

$\endgroup$
2
  • 1
    $\begingroup$ Convolutional layers are not really that new, they have been around since at least the late 1980s. Robust training algorithms, such as Rprop have been around for quite a while as well. I think a major factor that isn't currently on the list is simply the computational power available (I used to run my ANNs on a dual 133Mhz pentium machine with 32Mb of memory - which was so powerful at the time that my colleagues were just a little jealous! ;o) $\endgroup$ Dec 11, 2019 at 8:42
  • 1
    $\begingroup$ @Dikran More computing power, and GPU-based training. Even a 5GHz CPU has little on a medium-tier GPU when it comes to training CNNs. I also read elsewhere that for the longest time deep neural networks were deemed almost impossible to train due to instability, until layerwise unsupervised training became a thing. For a while this was thought to be the way to train deep networks, until better initialization schemes were available. I will add the source later if I can find it. $\endgroup$ Dec 11, 2019 at 11:10

Not the answer you're looking for? Browse other questions tagged or ask your own question.