1
$\begingroup$

I am working in a classification problem in which I use RBF with a single hidden layer. I want to use SoftMax activation function for the output layer. I already read some documents about the activation function of the output layer but can't find my answer. I try to write the equation for softmax and linear activation function and compare them. in my opinion yes they are different, but after I saw some documents I become confused so to become sure about I decide to ask it here.

is there any difference between linear and nonlinear activation function in a single hidden network?

$\endgroup$
2
$\begingroup$

is there any difference between linear and nonlinear activation function in a single hidden network?

If you have a single-layer network with activations functions that are all linear, the network is a linear model -- the same as a linear regression. This is because compositions of linear functions are still linear. If the last layer is a softmax, but the hidden layer is linear, this is a logistic regression.

If you have a single-layer network with nonlinear hidden units, it is not equivalent to any linear model. It is expressly a nonlinear model.

$\endgroup$
1
  • $\begingroup$ I have RBF network so the hidden layer is nonlinear (using Gaussian kernel) and I use softmax function for the output layer, so according to your answer using the linear output layer is not equivalent to any nonlinear activation function like softmax? $\endgroup$ – mkafiyan Aug 20 '18 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.