I'm interested in (Deep) Reinforcement Learning (RL). Before diving into this field should I take a course in Game Theory (GT)?
How are GT and RL related?
$\begingroup$
$\endgroup$
-
2$\begingroup$ They are about as closely related as hammers and whipped cream. You can probably find a problem where you can use both, but it's not common. $\endgroup$ – Don Reba Apr 21 '16 at 18:56
-
5$\begingroup$ @DonReba Not according to two well-known researchers in Reinforcement Learning: udacity.com/course/… I think that Game Theory tells you what's the optimum policy, while RL tells you how the agents can learn the optimum or a good policy. $\endgroup$ – Kiuhnm Apr 21 '16 at 19:19
-
3$\begingroup$ @DonReba, perhaps in terms of the usual content that is taught in them. However, the purposes of the two fields are not so different. Reinforcement learning could be looked at as a game of imperfect information, often for one player. Or as a two player game in which the other player, nature, follows a set of rules you wish to discover. $\endgroup$ – conjectures Apr 24 '16 at 7:21
-
1$\begingroup$ This was educational. :) $\endgroup$ – Don Reba Apr 24 '16 at 8:03
$\begingroup$
$\endgroup$
In Reinforcement Learning (RL) it is common to imagine an underlying Markov Decision Process (MDP). Then the goal of RL is to learn a good policy for the MDP, which is often only partially specified. MDPs can have different objectives such as total, average, or discounted reward, where discounted reward is the most common assumption for RL. There are well-studied extensions of MDPs to two-player (i.e., game) settings; see, e.g.,
Filar, Jerzy, and Koos Vrieze. Competitive Markov decision processes. Springer Science & Business Media, 2012.
There is an underlying theory shared by MDPs and their extensions to two-player (zero-sum) games, including, e.g., the Banach fixed point theorem, Value Iteration, Bellman Optimality, Policy Iteration/Strategy Improvement etc. However, while there are these close connections between MDPs (and thus RL) and these specific type of games:
- you can learn about RL (and MDPs) directly, without GT as a prerequisite;
- anyway, you would not learn about this stuff in the majority of GT courses (which would normally be focused on, e.g., strategic-form, extensive-form, and repeated games, but not the state-based infinite games that generalize MDPs).
$\begingroup$
$\endgroup$
Game theory is quite involved in the context of Multi-agent Reinforcement learning (MARL).
Take a look at stochastic games or read the article An Analysis of Stochastic Game Theory for Multiagent Reinforcement Learning.
I would not see GT as a prerequisite for RL. However, it provides a nice extension to the multi-agent case.
$\begingroup$
$\endgroup$
RL: A single agent is trained to solve a Markov decision problem (MDPS). GT: Two agents are trained to solve Games. A multi-agent Reinforcement learning (MARL) can be used to solve for stochastic games.
If you are interested in the single-agent application of RL in deep learning, then you do not need to go for any GT course. For two or more agents you may need to know the game-theoretic techniques.
