What is a "recurrent reinforcement learning"?
Recurrent reinforcement learning (RRL) was first introduced for training neural network trading systems in 1996. "Recurrent" means that previous output is fed into the model as a part of input. It was soon extended to trading in a FX market.
The RRL technique has been found to be a successful machine learning technique for building financial trading systems.
What is the difference between "recurrent reinforcement learning" and normal "reinforcement learning" (like Q-Learning algorithm)?
The RRL approach differs clearly from dynamic programming and reinforcement algorithms such as TD-learning and Q-learning, which attempt to estimate a value function for the control problem.
The RRL framework allows to create the simple and elegant problem representation, avoids Bellman's curse of dimensionality and offers compelling advantages in efficiency:
RRL produces real valued actions (portfolio weights) naturally without resorting to the discretization method in the Q-learning.
RRL has more stable performance compared to the Q-learning when exposed to noisy datasets. Q-learning algorithm is more sensitive to the value function selection (perhaps) due to the recursive property of dynamic optimization, while RRL algorithm is more flexible in choosing objective function and saving computational time.
With RRL, trading systems can be optimized by maximizing performance functions, $U( )$, such as "profit" (return after transaction costs), "wealth", utility functions of wealth or risk-adjusted performance ratios like the "sharpe ratio".
Here you will find a Matlab implementation of the RRL algorithm.
Reinforcement Learning for Trading
Reinforcement Learning for Trading Systems and Portfolios
FX trading via recurrent reinforcement learning
Stock Trading with Recurrent Reinforcement Learning (RRL)
Algorithm Trading using Q-Learning and Recurrent Reinforcement Learning
EXPLORING ALGORITHMS FOR AUTOMATED FX TRADING – CONSTRUCTING A HYBRID MODEL