I have built a deep reinforcement learning based portfolio optimisation agent. At a high level it is using macro economic data, valuations of the assets and a few technical indicators as the features. The policy network is a temporal convolution network with attention. The output of the policy network is the portfolio allocation. The agent is episodic with 365 days as the episode length. The episode terminates either at the end of of 365 days or when the drawdown of the portfolio hits 30%. With these conditions I have trained the agent on 10 years of data. The problem I am facing is of figuring when to stop training the agent. All the metrics like loss continue to improve with each iteration. The out of sample performance fluctuates in a narrow band after some 100 iterations indicating that it is possibly the best result the model can provide. But the loss metric keep decreasing.

Training Loss

Gradients of the Mode

enter image description here

Out of sample Validation Returns

Out of sample Validation Loss

As one can see from these graphs that one cannot make a decision on when to stop the training based on the training loss. And any decision cannot be made based on Out of Sample Validation data set. Any suggestions ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.