I had a regression problem with small data set, I solved it with neural networks (MLP, ELM,..) As convention, I used a linear function for output layer, the results were not so good. I tried to change to sigmoid for MLP and it gives me better results, even with ELM the sine function gives me better results.

But, when I read the literature on the choice of activation functions, many posts, almost if I can say "forbidden" the use of non-linear functions for the output layers, So what do you think?

  • $\begingroup$ You are doing regression, you do not really need an activation function in the end for this purpose, or if you do use one then a linear would be equivalent. Those other activation functions (sigmoid, sine, etc.) are for classification, it dosn't really make sense to use them in classification. $\endgroup$ May 13 at 5:48


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.