Is a loss function computed after each step of gradient descent or after a whole epoch? In neural networks with mini-batch or stochastic gradient descent, is a loss function computed after each step of gradient descent or after a whole epoch? 
 A: it can be done either way but more important is gradient and it also can be done either way, but one thing to clarify you should: start epoch, compute loss, then gradient(sgd step), then next epoch starts, loss etc.
A: The gradient of the loss must be computed each step. Without knowing the gradient of the loss, backprop can't update the parameters.
When you compute the value of the loss is up to you; most neural network libraries compute the loss value of the mini-batch "for free" when doing the forward pass of the network.
As an aside, computing the value of the loss for the whole training set at the end of an epoch can be useful, because SGD is noisy.
A: The training version of the loss function is computed during each minibatch for free on the minibatch of training data itself.  In contrast, the testing version of the loss function does not have to be executed at all, but people tend to set a regular schedule for it as a way to monitor or evaluate how the model is doing as it trains. 
For example, this evaluation schedule can be once an epoch, or maybe once every 100 steps, or once every step, depending on your preference. The testing version of the loss function may or may not be the same; some commonly used components like batch normalization or dropout behave slightly differently between training and testing configurations. Also, typically, when evaluating, you would want to execute the testing on specifically held-out testing data.
