1
$\begingroup$

Say we are transferring sequentially from environment 1-3 below, where the text corresponding to each environment describes its observation space.

Env 1 observation: position of robot

Env 2 observation: position of robot, position of object 1, velocity of object 1

Env 3 observation: position of robot, position of object 1, velocity of object 1, position of object 2

How do we setup the architecture for the value/policy network such that it can handle variable length observations like this?

$\endgroup$
  • $\begingroup$ It is not clear what the relation between Env 1, Env 2, etc. and the "variable length observations" is. Also, what do you mean by value/policy network? Edit your question to clarify this. $\endgroup$ – nbro Feb 6 at 17:14
0
$\begingroup$
  1. Define a fragment embedding network $E(x, l; \theta)$ which takes as input a small fragment of the state $x$ (such as the position), the type of information given $l$ (position, or velocity, etc) and outputs some $d$ dimensional vector.

  2. Choose some aggregation function $g$ which combines an arbitrary number of embeddings into a fixed size representation. This can be simple as $g(x_1, \ldots x_n) = \frac{1}{\sqrt{n}} \sum x_i$, or perhaps an LSTM $g(x_i; \phi)$.

  3. Take the output of $g$ as your state embedding.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.