# Transfer with different sized state spaces in neural networks/deep reinforcement learning

Say we are transferring sequentially from environment 1-3 below, where the text corresponding to each environment describes its observation space.

Env 1 observation: position of robot

Env 2 observation: position of robot, position of object 1, velocity of object 1

Env 3 observation: position of robot, position of object 1, velocity of object 1, position of object 2

How do we setup the architecture for the value/policy network such that it can handle variable length observations like this?

• It is not clear what the relation between Env 1, Env 2, etc. and the "variable length observations" is. Also, what do you mean by value/policy network? Edit your question to clarify this. – nbro Feb 6 at 17:14

1. Define a fragment embedding network $$E(x, l; \theta)$$ which takes as input a small fragment of the state $$x$$ (such as the position), the type of information given $$l$$ (position, or velocity, etc) and outputs some $$d$$ dimensional vector.
2. Choose some aggregation function $$g$$ which combines an arbitrary number of embeddings into a fixed size representation. This can be simple as $$g(x_1, \ldots x_n) = \frac{1}{\sqrt{n}} \sum x_i$$, or perhaps an LSTM $$g(x_i; \phi)$$.
3. Take the output of $$g$$ as your state embedding.