Using non-normalized data for learning a RL agent using PPO i want to use the baselines code from OpenAI to apply to a power trading setting  where I trade energy on a market. My observation space includes several kinds of data, which is why I originally used their spaces.Dict type. That however doesn't seem to work, they only accept an array of min/max bounded numpy arrays. 
I could just flatten my whole data and pass it in as an array. But some values are very small (typical between 0 and 2) while others go up into the thousands. 
What would be a good way to preprocess the data? Do I normalize it just before I feed it to the NN? What impact does that have on the networks learning ability? The data includes:


*

*historical average prices for the last 168 steps

*known prices and volumes for the currently open for trading timeslots (24 at a time)

*predictions of demand

*already purchased amounts in previous trading slots


code repo just in case
It's really a bummer that OpenAI baselines work so sparsely, I worked a long time on making this "Gym API compatible" but now I am unsure if their code will actually live much longer..
 A: Reinforcement learning does not itself require normalised state or action data. However, the RL context does not change neural network behaviour in this respect. Neural networks work better with normalised data. 
So, yes, the advice should be to normalise the data. You could either do that as part of state representation, or just before any input to the neural network model. What will work for you depends on the framework you are using. I suspect if you are using OpenAI's baselines that you will need to adjust the state representation. Although I am not sure, I have not used it, I took a quick look at baselines/ppo1/mlp_policy.py but there is too much abstraction for me to quickly assess how the state representation is processed before getting to the NNs.
It is unlikely at the outset that you will know the population of state and action representations under an optimal policy well enough to normalise by mean and standard deviation, so you may need to pick some other range adjusting approaches, or simply guess suitable multipliers and offsets. Normalisation does not need to be perfect, for NNs you just want to avoid being out by a significant factor.
If a specific input scales over a few orders of magnitude, and this is an important factor, then maybe work with (a multiple and offset of) the log of that input.
