Skip to main content
added 155 characters in body
Source Link
Skander H.
  • 12.1k
  • 2
  • 44
  • 99

Does the universal approximation theorem for neural networks hold for any activation function (sigmoid, ReLU, Softmax, etc...) or is it limited to sigmoid functions?

Update: As shimao points out in the comments, it doesn't hold for absolutely any function. So for which class of activation functions does it hold?

Does the universal approximation theorem for neural networks hold for any activation function (sigmoid, ReLU, Softmax, etc...) or is it limited to sigmoid functions?

Does the universal approximation theorem for neural networks hold for any activation function (sigmoid, ReLU, Softmax, etc...) or is it limited to sigmoid functions?

Update: As shimao points out in the comments, it doesn't hold for absolutely any function. So for which class of activation functions does it hold?

Source Link
Skander H.
  • 12.1k
  • 2
  • 44
  • 99

Does the universal approximation theorem for neural networks hold for any activation function?

Does the universal approximation theorem for neural networks hold for any activation function (sigmoid, ReLU, Softmax, etc...) or is it limited to sigmoid functions?