# What are the differences between contextual bandits, actor-citric methods, and continuous reinforcement learning?

Let's imagine we have a blackbox function f(X) -> y which we don't know. X is a vector of 10 continuous variables, which we want to optimize to reduce y output (a continuous reward with unknown min and max). Also, f(X) maps a different optimum y depending on some other continuous parameters Z, which are observable. Each day we can test a new X combination, and at the end of it we know the result.

I could find out three possible solutions to this problem:

In the case where some Z values are influenced by X (let's say only 2 non-related binary values), and other are external (3 real valued), which RL algorithm would fit better this problem, and how would I add these independent Z values to take them into account?

If all Z values are external, is multiarmed bandits the best approach?

• None of the reinforcement learning or bandit-solving algorithms are really suitable for your black box problem. The bandit problem solver is maybe closest if $Z$ varies and is observable. You don't have a reinforcement learning problem unless choosing $X$ has a direct impact on the next value of $Z$. Commented Jan 9, 2018 at 8:14
• Let's say all the Z varies and are observable, all of them have an impact on the reward, but only some of them are influenced by X. In this case, which of these algorithms (or other) would be more suitable? I edited the question Commented Jan 9, 2018 at 8:37
• I would say the question is valid from the information acquisition perspective. Commented Jan 11, 2018 at 0:54
• @freesoul have you considered Gaussian process bandits? The variables Z can encode information such as smoothness of the function and the framework handles continuous x naturally. Commented Jan 14, 2018 at 18:20

The core idea is to systematically try suboptimal x which shall pay off in the long term.