Single loss value for gradient descent in neural network optimization

Suppose I train 2 neural networks in stock trading. First network produces enter-trade signals (very sparse). The second network produces exit-trade signals and starts to give signals right after the first gives positive signal.

Question is whether it is mathematically sane to get gradient using just one loss value, which is a function of a squared difference of cumulative return (after 10K trades, for example) from theoretical cumulative return. Doing this trick for both networks, like sharing gradient.

E.g., L <- (-5000 - 20000)^2.

Clarification example of what I meant:

Suppose neural networks have been randomly initialized.

sum_ret_theoretical <- 20000 # theoretical cumulative return

price, input_1, input_2, neural_output_1, neural_output_2
1.000, 0.1, 0.2, 0, NA
1.001, 0.3, 0.1, 0, NA
1.007, 0.9, 0.2, 1, NA

Here first NN gave positive signal

1.01, 0.1, 0.2, NA, 0
1.016, 0.3, 0.1, NA, 0
1.03, 0.9, 0.2, NA, 1

Here second NN gave positive signal

return(i) = 1.03 - 1.007 - 0.0001 = 0.0229 # (trade gained a positive outcome after excluding a constant cost).

Go through all examples [1 in 1:I] until the end of the sample, accumulate returns in vector Ret. Here first training episode has ended.

sum_ret <- sum(Ret)

L <- (sum_ret - sum_ret_theoretical) ^ 2 # get loss function value after first episode ends

calculate neural networks' updates w.r.t. to L derivation (gradients) ## that is the question.


Can I train NN based on extremely sparse loss values (1 per epoch/episode) that is a function of accumulated NN performance. An example of that maybe a NN that is designed to approximate the sum (mean) of the sample of values from variable X, without looking at each x to get loss.

• Two neural networks process thousands of examples, producing scalar predictions. However, for each example there is no label (no way to get loss per sample). The predictions are instead used to get 1 scalar loss after whole sample has been processed. That makes just one loss function value per iteration. Is it enough to calculate gradient descent routine? And if so, can it be shared between both networks that contribute to the loss function value? Commented Jun 12, 2018 at 13:53
• @JanKukacka, please look at edited question. I made it clearer I hope. Commented Jun 13, 2018 at 10:49

Notes: if you have two NN who train on the same features, you can make a single one with a two row vector prediction. Also instead of having a loss function converging to sum_ret_theoretical, maybe you could tweak it to have a score to maximize.

Case 1: If you do stochastic gradient descent (batch size = 1), the loss function is either undefined or 0.

Case 2: If your batch size > 1, your loss function can be undefined if neural_output_1 = 1 after neural_output_2 = 1.

Therefore I believe you cannot achieve it.

To deal with temporal data, check out recurrent neural networks. To maximize gain, check out (deep) reinforcement learning.

Here, it looks like you're trying to train with data from the future, which bidirectional RNN could handle. But if you're planning to trade, you can't use data from the future.

• thank you for your notes. Let me clarify a little bit. The 2 NNs have sligtly different features in input space: first works with price data mostly, and second works with both prices and state-of-trade information. They can also have different neuron number / architecture, but I am not sure how different. "If you do stochastic gradient descent (batch size = 1), the loss function is either undefined or 0." Of course I will not run SGD, I will run whole sample to accumulate full trade sequence. Commented Jun 14, 2018 at 9:27
• "If your batch size > 1, your loss function can be undefined if neural_output_2 = 1 after neural_output_1 = 1". That would be 1 trade (signal equals 1 means that market was entered, market was exited). Commented Jun 14, 2018 at 9:30
• "Therefore I believe you cannot achieve it.". I don't really think so. I am thinking in 2 directions: first) get gradient numerically using L value and differentiating by each of the weights (so, doing lots of run-throughs for just one NN update), second) using the methods of optimization that are derivative-free. What do you think? "Here, it looks like you're trying to train with data from the future, which bidirectional RNN could handle." I am not sure this addresses my question properly. Commented Jun 14, 2018 at 9:32
• @Alexey Burnakov my mistake, I meant neural_ouput_1 = 1 after neural_ouput_2 = 1, I edited. This is the point making me believe it is not achievable. To update weights you will need $\frac{\partial L}{\partial a}$ but it might be undefined and sometimes $L(a)$ is not continuous. Commented Jun 14, 2018 at 11:11
• that is easily mitigated by not allowing the first NN to make predictions when it had made a signal and the second one has not yet (that is why I included NAs in the example table). And as I illustrated the signals are quite sparse, which can result in for example 100 returns per 10,000 examples. Commented Jun 14, 2018 at 12:08