Is it possible to train a Neural Network such that each of the last layer's nodes train with a different Loss or Objective function?
This seems possible to me. You would simply update each node's weights with gradients calculated on using different loss function. Is this intuition correct?
To simplify the problem, let us assume this is a regression problem where the loss functions are similar.
Finally, has there been an research into this? I have looked but cannot find anything on the subject. Perhaps I am not searching for the right things.
NB: The reason I ask is it would be interesting to train a model with MAE on all but the final output layer and then have 3 output nodes where the loss function as MAE and then the Pin Ball Losses for 0.9 and 0.1 quantiles. MAE is just the 0.5 quantile loss so I assume the 3 output nodes to be similar and benefit from the body of the network.