4
$\begingroup$

Can all neural network having directed acyclic graph (DAG) topology be trained by back propagation methods? You can assume that the activation functions of all neurons are differentiable.

I mean by the gradient based methods like Stochastic gradient decent, AdaGrad,Adam, etc.

If it's true, is there a reference (academic paper is the best) providing the proof of it?

$\endgroup$
2
$\begingroup$

Can all neural network having directed acyclic graph (DAG) topology be trained by back propagation methods? I mean by the back propagation methods like Stochastic gradient decent, AdaGrad, Adam, etc.

The methods you mention are gradient-based, and subsequently won't work if one activation function used by the artificial neurons isn't differentiable. However, they are some ways around, e.g. using reinforcement learning.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I updated the question to add the assumption that all activation functions are differentiabla $\endgroup$ – rkjt50r983 Sep 13 '16 at 22:01
  • $\begingroup$ Reinforcement learning says it uses Back-Propagation for weights update... $\endgroup$ – HAL9000 May 30 '18 at 9:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.