1
$\begingroup$

I am new to reinforcement learning, i have studied about its basics and principles, however, i have one question:

If I implemented q-learning to a robot, for example, to make my robot learn to navigate around the house to pick up a ball, then the optimal policy for the robot in that house is found

If i am to change the environment, such as, moving the ball to another place, change the place of my furniture, or make the robot find a ball in my friend's house instead

Can the robot adapt to the new environment, and how fast will it be? Thank you

$\endgroup$
3
$\begingroup$

This depends on many things, but generally, it is difficult for Q-learning to just "adapt" to a new environment. By adapt, I assume you expect the robot to be able to find the ball without retraining the entire system. Obviously, the most fail safe way to make the algorithm work in any new scenario is to retrain the entire system.

The encoding of the state of the environment can be critical. For example, if you encode the state as the $(x,y)$ coordinate position of the robot, then it will not adapt at all when you change the position of the ball. This is because the Q-value of the original position of the ball was probably quite high, but by moving the ball, you have completely changed the Q-values of the game, and using the old Q-values will not make sense.

On the other hand, if you have a very sophisticated encoding of the state, including say, a 3D point cloud representing the world and an integrated history of the visual inputs from a camera, it may be possible to train a Q-learning algorithm in a way so that it can adapt to the more diverse examples in your post.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.