Suppose I have a regression model that can make predictions on stock price movements for 10 steps ahead. The labels are {"Price moves down": -1, "Price is stable": 0, "Price moves up": 1}. So at each time step, e.g. 0.5 seconds, the model spits an array of labels $[L_{t+1}, L_{t+2}, L_{t+3}, L_{t+4}, L_{t+5}, L_{t+6}, L_{t+7}, L_{t+8}, L_{t+9}, L_{t+10}]$

Now the idea is to use that model to train a deep reinforcement learning agent which can place limit buy and limit sell orders on the market. I am not interested in using that model for high frequency trading, but day trading instead. Here is an example where the agent might want to place the buys and sell orders.

enter image description here

Assume the features and the model regression predictions are well suited for the RL phase. How can I use the model so that the agent can use it to take a decision? Do I need to pass the predictions as well as the features inside the state variable? I thought feeding the agent with all the features as well as the price movement prediction at each time-step.

To accomplish the task correctly, the agent needs the right reward function. So to overlook the noisy up and down movements and keep only the long relevant day trading movements (look the above picture), I thought setting up a threshold on each transaction profit. Am I on the right track? In other words, once a pair (limit buy order, limit sell order) is filled, the profit needs to reach the threshold, otherwise the reward is set to 0. The python function might be

def reward(min_profit, current_profit):
    if current_profit < min_profit:
       return 0
       return current_profit

Is such a reward function appropriate?


1 Answer 1


Am I on the right track?

Sorry, but I can't get your idea. Firstly, the reward is determined by profit and transaction cost together. So you need to consider the cost ratio.

Secondly, why do you need to set up a threshold on each transaction profit? The negative profit is explicit signal for the agent.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.