0
$\begingroup$

In backwards propagation, one tries to minimize the cost function in the most efficient way by looking at small changes in the bias, the previous activation, and the weights of a neuron. We start in the output layer and work backwards towards the input layer. Is the cost function usually only defined using the output layer? Is it possible to define a cost function that uses a combination of the output layer and other layers (e.g. various hidden layers or even the input layer)?

In a sense, is "forward propagation" combined with "backwards propagation" possible?

$\endgroup$
1
$\begingroup$

Yes, it is typical that a loss function depends on more than just the output layer. For example with weight decay regularization, the loss is a function of all the weights in the network as well as the output.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.