How does Deep Reinforcement Learning remove the need to map or explore every state, action pair for an agent?

Question

I am interested in using Deep Reinforcement Learning to teach an AI how to play a game, where the AI knows the model of the game at the start (so I would use model-based deep reinforcement learning?)

But, the number of possible states and actions combinations that can be taken is very large, and I can't map every pair out. I heard that Deep Reinforcement Learning is a solution for this very large states space, but I'm not sure how exactly the Neural Net can be trained which action to take at any (future) state, if it hasn't experienced each possible state yet.

Could anyone please provide clarification on this subject?

Answer 1

Most forms of approximation in machine learning also lead to generalisation - the ability to give better-than-guesswork estimates for a target variable when presented with a previously unseen example.

Outside of RL, using a training dataset with a neural network or other function approximator, achieving this generalisation is the most common goal when training. This is the reason for cross-validation and test datasets, in order to measure how well the model has learned to generalise.

Deep RL, when exploring a very large state/action space, relies on this generalisation effect in order to learn effectively.

It can still be hard for an approximator to generalise well in board games where a very small difference in state can lead to radically different results. Hence self-playing learning systems like AlphaZero use complex architectures and significant compute resources to gain large amounts of experience (millions of games) in a small amount of time. This still falls far short of brute-forcing all possible states (by many orders of magnitude), so does still heavily rely on generalisation.