If I train two agents, one on environment A and one on environment A', where A' is just environment A padded with 10 rows of zeros, what can I predict will happen in terms of relative sample complexity of each agent?

Let's say the algorithm is soft actor critic.

In general, what principle would we use to think about this?

  • $\begingroup$ I don't understand the question. How is the environment represented? Is it a gridworld environment? $\endgroup$ – Philip Raeisghasem Mar 14 at 3:12

Since the padded entries are 0, then the gradient of the output of the actor and critic wrt any weights associated connected to those padded states is also 0, so it will be equivalent to the unpadded environment. In other words, the sample complexity will be the same.

If the padded entries are non-zero, they effectively are equivalent to redundant bias terms, which also doesn't affect the sample complexity.

  • $\begingroup$ What if instead of a constant padding, we add noisy random pixels that have no effect on reward? $\endgroup$ – user3180 Mar 14 at 5:54
  • $\begingroup$ @user3180 well then you will increase the sample complexity $\endgroup$ – shimao Mar 15 at 0:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.