Normally a DQN, uses a neuronal network to estimate the Q-Value. I have framed my problem as a regression problem before and have observed that XGBoost does outperform a NN.
Is it possible to replace the NN in a DQN with XGboost? I haven't found any hint/documentation about this at all. So anything would help me.
The downside of using XGBoost compared to a neural network, is that a neural network can be trained partially whereas an XGBoost regression model will have to be trained from scratch for every update. This is because an XGBoost model uses sequential trees fitted on the residuals of the previous trees so iterative updates to the model are not really possible. Someone did attempt this but didn't get very good results: but I don't see why it could not work in theory.
In Q learning, it is possible to use practically any regression model that can be updated incrementally.
In fitted Q learning, any regression model can be used, including tree-based approaches, see e.g. page 70 of this book: https://orbi.uliege.be/bitstream/2268/27963/1/book-FA-RL-DP.pdf
However, we can hardly speak about Deep Q learning that relates to deep neural networks by nature, i.e. including automated feature extraction from non-trivial objects (images, texts).
