"Forward Propagation" in Neural Networks

Question

In backwards propagation, one tries to minimize the cost function in the most efficient way by looking at small changes in the bias, the previous activation, and the weights of a neuron. We start in the output layer and work backwards towards the input layer. Is the cost function usually only defined using the output layer? Is it possible to define a cost function that uses a combination of the output layer and other layers (e.g. various hidden layers or even the input layer)?

In a sense, is "forward propagation" combined with "backwards propagation" possible?

Answer 1

Yes, it is typical that a loss function depends on more than just the output layer. For example with weight decay regularization, the loss is a function of all the weights in the network as well as the output.