In the case of a neural network,are there different cost functions for different hidden layers? or is there one cost function for the final layer ? For example, in the neural network, the hidden layer has sigmoid and Relu activations, and the output layer has linear activation. So, Will the cost function be MSE depending on the final output layer's activation? or Will there be different cost function for different layer- hidden Cost function log loss for sigmoid activation in hidden layer ?
$\begingroup$
$\endgroup$
0
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
Is there hidden cost function for hidden layer in the neural network?
Nearly all neural networks have loss functions only at the final layer. All weights and biases are updated by using backpropagation, which computes the loss on the final layer and uses the gradient of the loss wrt the weights and biases to adjust them.
The exceptions to this are explicit about having auxiliary loss functions sprinkled throughout, but this is a very specialized case. There are several examples in What is auxiliary loss as mentioned in PSPNet paper. These include:
- "Going Deeper with Convolutions" (2014) by Christian Szegedy et al.
- "Pyramid Scene Parsing Network" (2017) by Hengshuang Zhao et al.
