A somewhat naive question about MDP and RL:
Usually one assumes that the next environment state depends on the action the agent chooses. E.g. this is clearly the case in games like go. But what if it doesn't? Will the whole RL degenerate into something simpler? Is there a good reference for that?
For example, assume that in atari breakout the reflection of the ball doesn't depend on the position or velocity - one still has learn, that escaping through the side door is the best.