I have a series of 3-4 attributes (call them $X_1$, $X_2$, $X_3$) which can all have values of, say, any number on $[1,100]$. What I'd like to do is, first, build a group of decision trees (call them $T_1, T_2,..., T_n$) and weight them using a training set. Then, depending on how well each tree is doing at a given time on a given day, as data is fed in (thereby changing the attribute values), I'd like to use some form of online reinforcement learning to dynamically adjust the weights of each tree. Is this possible? I come from a non-Statistics background so I'm not familiar with the current literature in an area like this. Any help would be great!
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If you want to employ a reinforcement learning algorithm on this problem you'd probably encode the features as state, the weights as actions and the classification loss as reward. The actions should be continuous so a policy search would be better. Take a look at policy gradient as well as gradient free methods such as RL with CMA-ES or Trust region policy optimization. Approaches that are more like black-box optimization are likely better because you don't really have a value function.
Which brings me to the second point: reinforcement learning is likely to general as a framework for your problem. Your actions don't influence your state and your reward is observed immediately. Furthermore, you can fully observe the classification loss for all actions in a given state. I'm not an expert on the subject but to me it sounds like contextual online optimization. Take a look at the literature in that field as well. Given the nature of the online optimization community, they'll likely have strong theoretical guarantees if you can find an algorithm that applies. I have only skimmed it very briefly, but [1] seems like a good read. (Context-FTPL in particular). However, the online optimization literature is fairly dense.
It also sounds like someone might have done something like this before but I'm not aware of it.
[1] Syrgkanis, Vasilis, Akshay Krishnamurthy, and Robert E. Schapire. "Efficient Algorithms for Adversarial Contextual Learning." arXiv preprint arXiv:1602.02454 (2016).
