4
$\begingroup$

A somewhat naive question about MDP and RL:

Usually one assumes that the next environment state depends on the action the agent chooses. E.g. this is clearly the case in games like go. But what if it doesn't? Will the whole RL degenerate into something simpler? Is there a good reference for that?

For example, assume that in atari breakout the reflection of the ball doesn't depend on the position or velocity - one still has learn, that escaping through the side door is the best.

$\endgroup$
3
$\begingroup$

I'm not sure Atari Breakout is a good example of this. Your action, moving the paddle, definitely affects the state of the game.

However, if you're interested in "RL without states" then you should look into the Multi-armed Bandit setting. The idea is you need to choose a sequence of actions that maximize your reward. Taking an action does not have any influence over future rewards. It is essentially a pure explore-exploit problem, without the added complexity of state transitions.

$\endgroup$
1
$\begingroup$

I believe in that case one can solve the problem by applying supervised learning and then choosing the optimal action separately while achieving optimality.

I think in this example you present, the position of the ball is part of the environment state.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.