I find two different type of Contextual Bandit problem formulations in the literature:
Definition 1: (https://hunch.net/~jl/projects/interactive/sidebandits/bandit.pdf) In a contextual bandits problem, there is a distribution $P$ over $(x,r_1,...,r_k)$, where x is context, $a \in \{1,...,k\}$ is one of the k arms to be pulled, and $r_a \in [0, 1]$ is the reward for arm $a$. The problem is a repeated game: on each round, a sample $(x, r_1, ..., r_k)$ is drawn from $P$, the context $x$ is announced, and then for precisely one arm a chosen by the player, its reward $r_a$ is revealed.
Definition 2: (http://rob.schapire.net/papers/www10.pdf) The algorithm observes the current user $u_t$ and a set $A_t$ of arms or actions together with their feature vectors $x_{t,a}$ for $a \in A_t$. The vector $x_{t,a}$ summarizes information of both the user ut and arm $a$, and will be referred to as the context. Based on observed payoffs in previous trials, A chooses an arm $a_t ∈ A_t$, and receives payoff $r_t,a_t$ whose expectation depends on both the user $u_t$ and the arm $a_t$.
The fact that when stating definition number 2, the authors of http://rob.schapire.net/papers/www10.pdf cite the paper from the first definition is very confusing to me. In particular, they say "Following previous work [18], we call it a contextual bandit.1".
In particular, in Definition 1 assumes that only one context is revealed to the learner. In the second formulation, you observe "contexts" or better features for all the arms. I was thus wondering if there is any equivalence between the two formulations or a way to relate them.
All type of suggestion will be very appreciated. Thanks a lot for your help.