What's the relation between game theory and reinforcement learning? I'm interested in (Deep) Reinforcement Learning (RL). Before diving into this field should I take a course in Game Theory (GT)?
How are GT and RL related?
 A: Game theory is quite involved in the context of Multi-agent Reinforcement learning (MARL). 
Take a look at stochastic games or read the article An Analysis of Stochastic Game Theory for Multiagent Reinforcement Learning.
I would not see GT as a prerequisite for RL. However, it provides a nice extension to the multi-agent case. 
A: In Reinforcement Learning (RL) it is common to imagine an underlying Markov Decision Process (MDP). Then the goal of RL is 
to learn a good policy for the MDP, which is often only partially specified. MDPs can have different objectives such as total, average, or discounted reward, where discounted reward is the most common assumption for RL. There are well-studied extensions of MDPs to two-player (i.e., game) settings; see, e.g.,
Filar, Jerzy, and Koos Vrieze. Competitive Markov decision processes. Springer Science & Business Media, 2012.
There is an underlying theory shared by MDPs and their extensions to two-player (zero-sum) games, including, e.g., the Banach fixed point theorem, Value Iteration, Bellman Optimality, Policy Iteration/Strategy Improvement etc. However, while there are these close connections between MDPs (and thus RL) and these specific type of games:


*

*you can learn about RL (and MDPs) directly, without GT as a prerequisite;

*anyway, you would not learn about this stuff in the majority of GT courses (which would normally be focused on, e.g., strategic-form, extensive-form, and repeated games, but not the state-based infinite games that generalize MDPs).

A: RL: A single agent is trained to solve a Markov decision problem (MDPS).
GT: Two agents are trained to solve Games. A multi-agent Reinforcement learning (MARL) can be used to solve for stochastic games.
If you are interested in the single-agent application of RL in deep learning, then you do not need to go for any GT course. For two or more agents you may need to know the game-theoretic techniques.
A: If you already know game theory you may see many parallels with reinforcement learning with multiple agents. Decision theory which is essentially game theory with one player is a second area that similar and perhaps a closer match for single agent settings.
Strict Domination and Backwards Induction solution concepts map to the main steps used in the rl policy iteration algorithm which does a value function BI estimation followed by greedification step SD
The fields seem to diverge in some respects, game theory tends to makes strong assumptions on strategies being fully specified ahead of time and players being rational even when faced with an infinite decision tree. RL looks to solving these online and with minimal computational resources.
In this sense rl offers a more realistic and algorithmic approach to solving essentially the same class of problems.
The MDP Markov decision process used to formalise rl makes use of the markov property on the state which means rl decisions are made based on the current state and lack memory of earlier actions [unless explicitly included in the state].
A: RL environment can be modelled using MDP(Markov Decision Process), in case you are dealing with one single agent. If the environment consists of multiple agents, in this case it is called MultiAgent RL (MARL), then Game Theory (GT) may help. GT is used with MARL when there exists any sort of competition between the agents. But, if your agents will be fully cooperative (meaning that the agents cooperate together to achieve a common goal), in this case you may need to find some other approaches for coordination.
MARL settings are divided into three categories:

*

*Competitive in which the agents have conflicting interests; each wants to maximize its own reward (e.g. , minmax algorithm in game theory).

*Cooperative in which all the agents coordinate and cooperate to maximize a shared reward.

*Mixed mode, it is hybridized from both.

