I've read some articles about trading using recurrent reinforcement learning such as this one. The point where I do not fully understand is how to construct the cost/loss function.

In the article, Sharpe Ratio is one of the options that we can let the RNN minimize. The definition of Sharpe Ratio is $\frac{Average(R_t)}{StandardDeviation(R_t)}$ where $R_t$ is the return on investment. So I assume this $R_t$ (return) here is the reward of reinforcement learning.

The target of the algorithm is to maximize sharpe ratio, so my question is, how should I construct the structure of neural network/reinforcement learning framework in order to implement this gradient ascent method to maximize sharpe ratio?

In particular, the input data is price series, what should the output data be? What should the cost/loss function be?

  • $\begingroup$ Typically one seeks to maximize the Sharpe, but to each his own. $\endgroup$ – shabbychef Feb 2 at 5:44

If you split your dataset in investment periods (say you will make a bet every 5mins or 1day or other) I would imagine the output data is derived from the forward return in the next period.

In other words at the beginning of every investment period you need to open a position which you will close at the end of the period. Therefore the profit & loss for this period (pnl) will be proportional to the return of the asset times the size of your bet (you can consider for example that the size of the bet will be proportional to the intensity of your signal)

  • 1
    $\begingroup$ so that's one single data for each investment period, while I want to maximize sharpe ratio, so I should collect multiple pnl so as to compute the sharpe ratio, then maximize it? $\endgroup$ – Kevin. Fang Oct 23 '18 at 2:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.